I'm trying to fetch data from a webpage, but if the page isn't available the program runs for a long time until it timeouts. I need it to try to get the webpage for 10 seconds and if it doesn't get a response in that time it returns null. How can I make it to work that way, please?
Here is how I get the data:
public int getThreadData( String address ) throws IOException{
String valueString = null;
URL url = new URL( "http://" + address + ":8080/web-console/ServerInfo.jsp" );
URLConnection urlConnection = url.openConnection();
urlConnection.setRequestProperty( "User-Agent",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3)
Gecko/20100401" );
BufferedReader br = new BufferedReader( new InputStreamReader
( urlConnection.getInputStream(), "UTF-8" ) );
String inputLine;
while ( ( inputLine = br.readLine() ) != null )
{
if ( inputLine.contains( "#Threads" ) )
{
valueString = inputLine.substring( inputLine.indexOf( "/b>" ) + 3 );
valueString = valueString.substring( 0, valueString.indexOf( "<" ) );
}
}
br.close();
return Integer.parseInt( valueString );
}
Have you tried setting the connection timeout like following:
urlConnection.setConnectTimeout(10000); // 10000 milliseconds
You should probably use a HTTP library (like Apache's HTTPClient) that can simplify these things vastly. If you were using HTTPClient, you would do something like this:
// Set the timeout to 20-seconds.
final HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 20 * 1000);
HttpConnectionParams.setSoTimeout(httpParams, 20 * 1000);
DefaultHttpClient httpClient = new DefaultHttpClient(cm, httpParams);
HttpPost postRequest = new HttpPost(URL);
HttpResponse response;
response = httpClient.execute(postRequest);
Related
When i run this code it gives me error xml stating---Problem processing POST request: unsupported Content-Type text/xml. I want to send an xml using httpClient to a URL which is a web service url
public class HTTPClientDemo {
public static void main(String[] args) {
try
{
String inputXML = "<Style>0206.ard</Style><BoardCode>I-175B</BoardCode><BoardDesc>I-175 B Kraft</BoardDesc><GrainDirection>Vertical</GrainDirection><Unit>mm</Unit><PrintSide>Inside</PrintSide><Length>100</Length><Width>70</Width><Depth>45</Depth>";
URL url = new URL( "http://egwinae002:4415/ws/wstest003" );
// URL url = new URL( "https://dzone.com/articles/using-java-post-block-xml-web" );
URLConnection con = url.openConnection();
// con.connect();
// specify that we will send output and accept input
con.setDoInput(true);
con.setDoOutput(true);
con.setConnectTimeout( 20000 ); // long timeout, but not infinite
con.setReadTimeout( 20000 );
con.setUseCaches (false);
con.setDefaultUseCaches (false);
// tell the web server what we are sending
con.setRequestProperty ( "Content-Type", "text/xml" );
OutputStreamWriter writer = new OutputStreamWriter( con.getOutputStream() );
writer.write( inputXML );
writer.flush();
writer.close();
// reading the response
InputStreamReader reader = new InputStreamReader( con.getInputStream() );
StringBuilder buf = new StringBuilder();
char[] cbuf = new char[ 2048 ];
int num;
while ( -1 != (num=reader.read( cbuf )))
{
buf.append( cbuf, 0, num );
}
String result = buf.toString();
System.err.println( "\nResponse from server after POST:\n" + result );
}
catch( Throwable t )
{
t.printStackTrace( System.out );
}
// Have done modification and added some lines in your code, hope it will work
URLConnection con = url.openConnection();
//Added this one to make your connection http
HttpURLConnection conn = (HttpURLConnection) con;
// con.connect();
// specify that we will send output and accept input
conn .setDoInput(true);
conn .setDoOutput(true);
conn .setRequestProperty("accept-charset", "UTF-8");
// tell the web server what we are sending
conn.setRequestProperty ( "Content-Type", "text/xml" );
// set data posting method
conn.setRequestMethod("POST");
// OutputStreamWriter writer = new OutputStreamWriter( conn.getOutputStream() );
PrintWriter writer = new PrintWriter(conn.getOutputStream());
writer.write( inputXML );
writer.close();
// reading the response
// InputStreamReader reader = new InputStreamReader( conn.getInputStream() );
BufferedInputStream reader = new BufferedInputStream(conn.getInputStream());
update : this is a duplicate,
i'm building a Proxy-custom-tag with grails taglib, per default it makes a get-request, now i'm facing the problem, that it should be able to handle Post-requests too,#
and i'm able to check the request-method and conditionally set the openConnection method to post if necessary, but i dont know how to append the post-params to the request.
here 's my code so far
def wordpressContent = { attrs, body ->
def url
def requestMethod = request.getMethod()
def queryString = request.getQueryString()?'&'+request.getQueryString():''
def content
println "method :"+requestMethod
println "params == "+params // <- inside here are the post-parameters
url = grailsApplication.config.wordpress.server.url+attrs.pageName+'?include=true'+queryString
try {
content = url.toURL().openConnection().with { conn ->
if(requestMethod == 'POST'){
println "Its a POST"
conn.setRequestMethod("POST")
conn.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
// HOW to append the params here ?
}
readTimeout = 6000
if( responseCode != 200 ) {
throw new Exception( 'Not Ok' )
}
conn.content.withReader { r ->
r.text
}
}
}
catch( e ) {
println "exception : "+e
content="<div class='float' style='margin-top:10px;width:850px;background-color:white;border-radius:5px;padding:50px;'>Hier wird gerade gebaut</div>"
}
out << content
}
im very stuck here right now, i found answers saying to use this syntax
Writer wr = new OutputStreamWriter(conn.outputStream)
wr.write(postParams)
wr.flush()
wr.close()
but i dont know how to include that to my existing code,
for any hints thanks in advance
update: my solution was to build up the post-parameter-querystring by iterating over the params object in this pattern "xyz=zyx&abc=cba" and write it to the outputStream like above
// HTTP POST request
private void sendPost() throws Exception {
String url = "https://selfsolve.apple.com/wcResults.do";
URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
String urlParameters = "sn=C02G8416DRJM&cn=&locale=&caller=&num=12345";
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + urlParameters);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
}
you are using grails so you can also use groovy HTTPBuilder like below
http://groovy.codehaus.org/modules/http-builder/doc/
i've tried grabbing html source of this webapge http://www.mindef.gov.sg/content/imindef/press_room/official_releases/nr/2013/jan/22jan13_nr.html . however i encounted error as it responded with a different type of html as compared to what i am able to see from the browser. it seems like doing a httopost to the web and on the app result in different type of respond
address="http://www.mindef.gov.sg/content/imindef/press_room/official_releases/nr/2013/jan/22jan13_nr.html";
String result = "";
HttpClient httpclient = new DefaultHttpClient();
// httpclient.getParams().setParameter("http.protocol.single-cookie-header", true);
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Linux; U; Android 2.2.1; en-ca; LG-P505R Build/FRG83) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1");
InputStream is = null;
HttpGet httpGet = new HttpGet (address);
HttpResponse response = httpclient.execute(httpGet);
HttpEntity entity = response.getEntity();
is = entity.getContent();
InputStream is = null;
Try :
URLConnection cn= new URL(url).openConnection();
BufferedReader input = new BufferedReader( new InputStreamReader( cn.getInputStream() ) );
Read input stream.
Sounds like you're getting a mobile version of the site. If you extend the URL to include ?siteversion=pc you should get the page as served to a browser on your computer.
Try this:
StringBuilder builder = new StringBuilder();
String line = null;
HttpGet get = new HttpGet("http://www.url.com");
HttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(get);
InputStream is = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
while ((line = reader.readLine()) != null) builder.append(line);
Then, the page source should be in builder.
I'm running a tomcat servlet on my local machine and an Android emulator with an app that makes a post request to the servlet. The code for the POST is below (without exceptions and the like):
String strUrl = "http://10.0.2.2:8080/DeviceDiscoveryServer/server/devices/";
Device device = Device.getUniqueInstance();
urlParameters += URLEncoder.encode("user", "UTF-8") + "=" + URLEncoder.encode(device.getUser(), "UTF-8");
urlParameters += "&" + URLEncoder.encode("port", "UTF-8") + "=" + URLEncoder.encode(new Integer(Device.PORT).toString(), "UTF-8");
urlParameters += "&" + URLEncoder.encode("address", "UTF-8") + "=" + URLEncoder.encode(device.getAddress().getHostAddress(), "UTF-8");
URL url = new URL(strUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
OutputStreamWriter wr = new OutputStreamWriter(connection.getOutputStream());
wr.write(urlParameters);
wr.flush();
wr.close();
Whenever this code is executed, the servlet isn't called. However if I change the type of the request to 'GET' and don't write anything to the outputstream, the servlet gets called and everything works fine. Am I just not making the POST correctly or is there some other error?
try the following code, it may help u
try
{
String argUrl =
"";
String requestXml = "";
URL url = new URL( argUrl );
URLConnection con = url.openConnection();
System.out.println("STRING" + requestXml);
// specify that we will send output and accept input
con.setDoInput(true);
con.setDoOutput(true);
con.setConnectTimeout( 20000 ); // long timeout, but not infinite
con.setReadTimeout( 20000 );
con.setUseCaches (false);
con.setDefaultUseCaches (false);
// tell the web server what we are sending
con.setRequestProperty ( "Content-Type", "text/xml" );
OutputStreamWriter writer = new OutputStreamWriter( con.getOutputStream() );
writer.write( requestXml );
writer.flush();
writer.close();
// reading the response
InputStreamReader reader = new InputStreamReader( con.getInputStream() );
StringBuilder buf = new StringBuilder();
char[] cbuf = new char[ 2048 ];
int num;
while ( -1 != (num=reader.read( cbuf )))
{
buf.append( cbuf, 0, num );
}
String result = buf.toString();
System.err.println( "\nResponse from server after POST:\n" + result );
}
catch( Throwable t )
{
t.printStackTrace( System.out );
}
Out of curiosity I tried getting the response code for my request:
int responseCode = connection.getResponseCode();
System.out.println(responseCode);
This actually made the request go through and I got 200. I checked my Tomcat logs and the request finally got handled. I guess doing url.openConnection() and writing to the OutputStream isn't enough.
I am writing the program in Java, and i want to fill out the specified fields of form, simulate submit clicking, so get the result page. I am testing my idea on the url http://stu21.kntu.ac.ir/Login.aspx that have two fields txtusername and txtpassword. I am trying the code as follow but it does not return the result page for me. How can i do it ? What do i wrong in this code ?
DefaultHttpClient conn = new DefaultHttpClient();
conn = new DefaultHttpClient();
ArrayList<NameValuePair> pairs = new ArrayList<NameValuePair>();
pairs.add(new BasicNameValuePair("txtusername", "8810103"));
pairs.add(new BasicNameValuePair("txtpassword", "8810103"));
HttpPost httpPost = new HttpPost("http://stu21.kntu.ac.ir/Login.aspx");
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(pairs,
"UTF-8");
httpPost.setHeader(
"UserAgent",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.151 Chrome/18.0.1025.151 Safari/535.19");
httpPost.setEntity(entity);
HttpResponse response = conn.execute(httpPost);
InputStream is = response.getEntity().getContent();
RandomAccessFile raf = new RandomAccessFile(
"/home/hossein/Desktop/random.aspx", "rw");
raf.seek(0);
int bytes = 0;
byte[] buffer = new byte[1024];
while ((bytes = is.read(buffer)) != -1)
raf.write(buffer, 0, bytes);
raf.close();
is.close();
Sorry if my question duplicates with another threads, i can not find my solution on other threads.
Thanks In Advance :)
I think you need HTTPUnit. There is a good tutorial at javaworld.
Just look at the following example there :
public void testGoodLogin() throws Exception {
WebConversation conversation = new WebConversation();
WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret" );
WebResponse response = conversation.getResponse( request );
WebForm loginForm = response.getForms()[0];
request = loginForm.getRequest();
request.setParameter( "name", "master" );
// "clicking the button" programatically
response = conversation.getResponse( request );
assertTrue( "Login not accepted",
response.getText().indexOf( "You made it!" ) != -1 );
assertEquals( "Page title", "Top Secret", response.getTitle() );
}
I am sure that you can do your testing just like this.