I am writing the program in Java, and i want to fill out the specified fields of form, simulate submit clicking, so get the result page. I am testing my idea on the url http://stu21.kntu.ac.ir/Login.aspx that have two fields txtusername and txtpassword. I am trying the code as follow but it does not return the result page for me. How can i do it ? What do i wrong in this code ?
DefaultHttpClient conn = new DefaultHttpClient();
conn = new DefaultHttpClient();
ArrayList<NameValuePair> pairs = new ArrayList<NameValuePair>();
pairs.add(new BasicNameValuePair("txtusername", "8810103"));
pairs.add(new BasicNameValuePair("txtpassword", "8810103"));
HttpPost httpPost = new HttpPost("http://stu21.kntu.ac.ir/Login.aspx");
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(pairs,
"UTF-8");
httpPost.setHeader(
"UserAgent",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.151 Chrome/18.0.1025.151 Safari/535.19");
httpPost.setEntity(entity);
HttpResponse response = conn.execute(httpPost);
InputStream is = response.getEntity().getContent();
RandomAccessFile raf = new RandomAccessFile(
"/home/hossein/Desktop/random.aspx", "rw");
raf.seek(0);
int bytes = 0;
byte[] buffer = new byte[1024];
while ((bytes = is.read(buffer)) != -1)
raf.write(buffer, 0, bytes);
raf.close();
is.close();
Sorry if my question duplicates with another threads, i can not find my solution on other threads.
Thanks In Advance :)
I think you need HTTPUnit. There is a good tutorial at javaworld.
Just look at the following example there :
public void testGoodLogin() throws Exception {
WebConversation conversation = new WebConversation();
WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret" );
WebResponse response = conversation.getResponse( request );
WebForm loginForm = response.getForms()[0];
request = loginForm.getRequest();
request.setParameter( "name", "master" );
// "clicking the button" programatically
response = conversation.getResponse( request );
assertTrue( "Login not accepted",
response.getText().indexOf( "You made it!" ) != -1 );
assertEquals( "Page title", "Top Secret", response.getTitle() );
}
I am sure that you can do your testing just like this.
Related
I have the following Java code to send a POST request to SharePoint REST API to create a list and it returns the following authentication errors:
CloseableHttpClient httpClient = null;
try {
String user = xxx;
String password = xxx;
String domain = xxx;
String workstation = "";
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(AuthScope.ANY),
new NTCredentials(user, password, workstation, domain));
httpClient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
String digestQueryURL = "http://my_sharepoint_site/_api/contextinfo";
HttpPost httpPost = new HttpPost(digestQueryURL);
httpPost.addHeader("Accept", "application/json;odata=verbose");
CloseableHttpResponse response = httpClient.execute(httpPost);
byte[] content = EntityUtils.toByteArray(response.getEntity());
String jsonString = new String(content, "UTF-8");
ObjectMapper mapper = new ObjectMapper();
JsonNode j = mapper.readTree(jsonString);
String formDigestValue = j.get("d").get("GetContextWebInformation").get("FormDigestValue").toString();
response.close();
// now try to create the list
String url = "http://my_sharepoint_site/_api/web/lists";
HttpPost httpPost2 = new HttpPost(url);
httpPost2.addHeader("X-RequestDigest", getFormDigest(httpClient));
httpPost2.addHeader("Accept", "application/json;odata=verbose");
httpPost2.addHeader("Content-Type", "application/json;odata=verbose");
String body = "{ '__metadata': { 'type': 'SP.List' }, 'AllowContentTypes': true, 'BaseTemplate': 100, 'ContentTypesEnabled': true, 'Description': 'My list description', 'Title': 'Test' }";
StringEntity se = new StringEntity(body);
httpPost2.setEntity(se);
CloseableHttpResponse response2 = httpClient.execute(httpPost2);
StringBuilder result = new StringBuilder();
System.out.println(response2.getStatusLine().toString());
BufferedReader br = new BufferedReader(new InputStreamReader(response2.getEntity().getContent()));
String output;
while ((output = br.readLine()) != null) {
result.append(output);
}
System.out.println(result.toString());
} catch (Exception e) {
}
Console output
HTTP/1.1 403 FORBIDDEN
{"error":{"code":"-2130575251, System.Runtime.InteropServices.COMException","message":{"lang":"en-US","value":"The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again."}}}
I can use very similar code to send GET requests to the REST API to retrieve all lists, retrieve list items, perform all these read operations. However this does not work for POST requests. Am I doing something wrong? The credentials provided are for an account that has full control over the entire site collection, so we can rule out permission errors.
Alright, the problem is really very simple. This line:
String formDigestValue = j.get("d").get("GetContextWebInformation").get("FormDigestValue").toString();
Returns the formDigestValue with quotation marks enclosing it. Using asText() instead of toString() helped.
I'm trying to fetch data from a webpage, but if the page isn't available the program runs for a long time until it timeouts. I need it to try to get the webpage for 10 seconds and if it doesn't get a response in that time it returns null. How can I make it to work that way, please?
Here is how I get the data:
public int getThreadData( String address ) throws IOException{
String valueString = null;
URL url = new URL( "http://" + address + ":8080/web-console/ServerInfo.jsp" );
URLConnection urlConnection = url.openConnection();
urlConnection.setRequestProperty( "User-Agent",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3)
Gecko/20100401" );
BufferedReader br = new BufferedReader( new InputStreamReader
( urlConnection.getInputStream(), "UTF-8" ) );
String inputLine;
while ( ( inputLine = br.readLine() ) != null )
{
if ( inputLine.contains( "#Threads" ) )
{
valueString = inputLine.substring( inputLine.indexOf( "/b>" ) + 3 );
valueString = valueString.substring( 0, valueString.indexOf( "<" ) );
}
}
br.close();
return Integer.parseInt( valueString );
}
Have you tried setting the connection timeout like following:
urlConnection.setConnectTimeout(10000); // 10000 milliseconds
You should probably use a HTTP library (like Apache's HTTPClient) that can simplify these things vastly. If you were using HTTPClient, you would do something like this:
// Set the timeout to 20-seconds.
final HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 20 * 1000);
HttpConnectionParams.setSoTimeout(httpParams, 20 * 1000);
DefaultHttpClient httpClient = new DefaultHttpClient(cm, httpParams);
HttpPost postRequest = new HttpPost(URL);
HttpResponse response;
response = httpClient.execute(postRequest);
i've tried grabbing html source of this webapge http://www.mindef.gov.sg/content/imindef/press_room/official_releases/nr/2013/jan/22jan13_nr.html . however i encounted error as it responded with a different type of html as compared to what i am able to see from the browser. it seems like doing a httopost to the web and on the app result in different type of respond
address="http://www.mindef.gov.sg/content/imindef/press_room/official_releases/nr/2013/jan/22jan13_nr.html";
String result = "";
HttpClient httpclient = new DefaultHttpClient();
// httpclient.getParams().setParameter("http.protocol.single-cookie-header", true);
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Linux; U; Android 2.2.1; en-ca; LG-P505R Build/FRG83) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1");
InputStream is = null;
HttpGet httpGet = new HttpGet (address);
HttpResponse response = httpclient.execute(httpGet);
HttpEntity entity = response.getEntity();
is = entity.getContent();
InputStream is = null;
Try :
URLConnection cn= new URL(url).openConnection();
BufferedReader input = new BufferedReader( new InputStreamReader( cn.getInputStream() ) );
Read input stream.
Sounds like you're getting a mobile version of the site. If you extend the URL to include ?siteversion=pc you should get the page as served to a browser on your computer.
Try this:
StringBuilder builder = new StringBuilder();
String line = null;
HttpGet get = new HttpGet("http://www.url.com");
HttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(get);
InputStream is = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
while ((line = reader.readLine()) != null) builder.append(line);
Then, the page source should be in builder.
Below is a snipped from a Java Client that connects to a website and uploads a file via the POST method. I have to reproduce this client in a Visual Studio environment, but I don't see any equivalent functions in the .NET environment for the setEntity() function used in the Java.
Everything I've found points to using this...
public void uploadFile(File uploadFile, String partner, String key,
String baseUrl,boolean isPartner) throws IOException {
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(
CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1
);
String url = baseUrl + "?" + (isPartner ? "partnerId" : "ori") + "="
+ partner.toUpperCase() + "&authKey="
+ key+ "&key="
+ key;
HttpPost httppost = new HttpPost(url);
MultipartEntity multipartEntity = new MultipartEntity();
ContentBody contentBody = new FileBody(uploadFile, "text/xml");
multipartEntity.addPart("dataFile", contentBody);
httppost.setEntity(multipartEntity);
HttpResponse response;
response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
if (resEntity != null) {
resEntity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
}
Everything I've found in Visual studio uses something like this below for the POST method. The WebRequest object has no obvious way of adding the parameters I need.
Dim request As WebRequest = WebRequest.Create("http://Test.com/import?partnerId=2&authKey=XdUa")
request.Method = "POST"
Dim postData As String = StrData
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
request.ContentType = "dataStr"
request.ContentLength = byteArray.Length
Dim dataStream As Stream = request.GetRequestStream()
dataStream.Write(byteArray, 0, byteArray.Length)
dataStream.Close()
Dim response As WebResponse = request.GetResponse()
Any guidance would be greatly appreciated. If my question is not clear, let me know, I'll try again.
You can add following code snippet to add the parameter
request.ContentType="application/x-www-form-urlencoded"
Dim postData As String = "name1="+value1+"&name2="+value2
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
Rest will remain same.
The url: http://www.teamliquid.net/replay/download.php?replay=1830 is a download link to a .rep file.
My question is: how to download this content in java knowing the name of the original rep file in order to save it with a defined prefix, like path/_.rep
//I was trying to run wget from java but I don't see how to get the original file's name.
Get the redirected URL,
http://www.teamliquid.net/replay/upload/coco%20vs%20snssoflsekd.rep
You can get the filename from this URL.
It's tricky to get the redirected URL. See my answer to this question on how to do it with Apache HttpClient 4,
HttpClient 4 - how to capture last redirect URL
EDIT: Here is a sample using HttpClient 4.0,
String url = "http://www.teamliquid.net/replay/download.php?replay=1830";
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
HttpContext context = new BasicHttpContext();
HttpResponse response = httpClient.execute(httpget, context);
if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK)
throw new IOException(response.getStatusLine().toString());
HttpUriRequest currentReq = (HttpUriRequest) context.getAttribute(
ExecutionContext.HTTP_REQUEST);
String currentUrl = URLDecoder.decode(currentReq.getURI().toString(), "UTF-8");
int i = currentUrl.lastIndexOf('/');
String fileName = null;
if (i < 0) {
fileName = currentUrl;
} else {
fileName = currentUrl.substring(i+1);
}
OutputStream os = new FileOutputStream("/tmp/" + fileName);
InputStream is = response.getEntity().getContent();
byte[] buf = new byte[4096];
int read;
while ((read = is.read(buf)) != -1) {
os.write(buf, 0, read);
}
os.close();
After running this code, I get this file,
/tmp/coco vs snssoflsekd.rep