Develop Reference

Assertion that value is set in map in java and testng or junit

I have a test written in java which contains a simple HTTP request sent to the server and then capture the server response.
Here I want a way to assert the console output of the response header that I am receiving from the server?
Example on one of the response header: if "Content-Length" = 12227 then its a pass or else it fails that test.
Here is my code:
public class ResponseHeaderLoggedInUser {
#Test
public void ResponseHeadersforUSA() {
try {
//System.setOut(myconsole);
URL obj = new URL("http://www.ivivva.com/?locale=en_US");
URLConnection conn = obj.openConnection();
//Setting Request Headers//
conn.setRequestProperty("Host", "www.ivivva.com");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0");
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
conn.setRequestProperty("Accept-Encoding", "gzip, deflate");
conn.setRequestProperty("Cookie", "NSC_JOv4n55lbq2w4zpexoskqbexxcrzicc=ffffffff09d205ab45525d5f4f58455e445a4a423660; UsrLocale=en_US; Country=US; ca_ord=gGYBmQjaBCVj65TJ+vOBJw==; isLoggedin=true; cartCount=0; NSC_mvmvmfnpo_tubhfw10_jwjwwb_mc=ffffffff09d205ab45525d5f4f58455e445a4a4229a0; omniID=1482345027850BoER; mbox=session#1482345027884-6794#1482347607|check#true#1482345807; __utma=210616138.1023825236.1482345028.1482345028.1482345028.1; __utmb=210616138.5.10.1482345028; __utmc=210616138; __utmz=210616138.1482345028.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); s_cc=true; s_fid=752E92B2B20003FF-05DED4FECDE6B525; c_m2=1; c_m=undefinedDirect%20LoadDirect%20Load; s_nr=1482345746223; s_vnum=1484937028265%26vn%3D1; s_invisit=true; s_lv=1482345746225; s_lv_s=First%20Visit; s_sq=%5B%5BB%5D%5D; sandy-session-id=0fa3f19b82e69e2a; sandy-client-id=93c64748da80825e; BTT_X0siD=807679838813198396; BTT_Collect=on; s_vi=[CS]v1|2C2D6522051D272B-6000016360010F9A[CE]; _ga=GA1.2.1023825236.1482345028; sl=CA; BTT_WCD_Collect=off; JSESSIONID=61AE25BF6E33856CB6FDD67DC939D9B6; regionMsgShown=true; us_ord=27qFcsniy0Bv/8EVzdZzNw==; _gat_mobifyTracker=1; __utmt_ga=1");
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Upgrade-Insecure-Requests", "1");
conn.setRequestProperty("Cache-Control", "max-age=0");
Map<String, List<String>> map = conn.getHeaderFields();
System.out.println("Printing Response Header...\n");
System.out.println("This test is for Logged in user with userlocale present\n");
for (Map.Entry<String, List<String>> entry : map.entrySet()) {
System.out.println("Key : " + entry.getKey()
+ " ,Value : " + entry.getValue());
}
} catch (Exception e) {
e.printStackTrace();
}
}}

Look into using JUnit Test cases here. If you know how to retrieve the value of Content-Length, you can use the AssertEquals functionality, which takes the two parameters that you'd like to compare.
In the JUnit, your code would look something like this:
AssertEquals("Expected Value","Actual Value");
Thus, for your Content-Length:
AssertEquals("12227", map.get(Content-Length));
The only thing you need to be able to do is to retrieve the actual content-length correctly.

Using Java bufferedreader to get html from URL

I'm trying to read all the html from a page using a buffered reader like follows
String charset = "UTF-8";
URLConnection connection = new URL(url).openConnection();
connection.addRequestProperty("User-Agent",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)");
connection.setRequestProperty("Accept-Charset", charset);
InputStream response = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(response,charset));
then I'm reading it line by line like this:
String data = br.readLine();
while(data != null){
data = br.readLine();
}
the problem is I'm getting something like:
}$B!)(BL$B!)(Bu"~$B!)$(D"C(B|X$B!x!)!x(B}
I've tried this:
do {
data = br.readLine();
SortedMap<String, Charset> map = Charset.availableCharsets();
for(Map.Entry<String, Charset> entry : map.entrySet()){
System.out.println(entry.getKey());
try {
System.out.println(new String(data.getBytes(entry.getValue())));
} catch (Exception e) {
e.printStackTrace();
}
}
}while(data!=null)
and I'm not getting any readable html in any of them. This really weird since it was working fine until this morning and I didn't change anything..
What am I doing wrong here? is it possible that something changed in the website I'm trying to read? please help.

The Server has changed his transfer mode to compressed data, what you can see in response header from server:
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Mon, 09 Mar 2015 09:34:41 GMT
Server:nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding
X-Powered-By:PHP/5.5.16-pl0-gentoo
As you can see the content encoding is set to gzip Content-Encoding:gzip.
So you have to decode the zipped content first:
GZIPInputStream gzis = new GZIPInputStream(connection.getInputStream());
BufferedReader br = new BufferedReader(new InputStreamReader(gzis,charset));
To view the headers of requests and responses you could use a network monitor (see Free Network Monitor).
Simpler is it to use the developer plugins integrated in most common browsers. Here is the documentation of Chrome DevTools, how to use the network tab: https://developer.chrome.com/devtools/docs/network

capture redirect destination JAVA [duplicate]

I am accessing web pages through java as follows:
URLConnection con = url.openConnection();
But in some cases, a url redirects to another url. So I want to know the url to which the previous url redirected.
Below are the header fields that I got as a response:
null-->[HTTP/1.1 200 OK]
Cache-control-->[public,max-age=3600]
last-modified-->[Sat, 17 Apr 2010 13:45:35 GMT]
Transfer-Encoding-->[chunked]
Date-->[Sat, 17 Apr 2010 13:45:35 GMT]
Vary-->[Accept-Encoding]
Expires-->[Sat, 17 Apr 2010 14:45:35 GMT]
Set-Cookie-->[cl_def_hp=copenhagen; domain=.craigslist.org; path=/; expires=Sun, 17 Apr 2011 13:45:35 GMT, cl_def_lang=en; domain=.craigslist.org; path=/; expires=Sun, 17 Apr 2011 13:45:35 GMT]
Connection-->[close]
Content-Type-->[text/html; charset=iso-8859-1;]
Server-->[Apache]
So at present, I am constructing the redirected url from the value of the Set-Cookie header field. In the above case, the redirected url is copenhagen.craigslist.org
Is there any standard way through which I can determine which url the particular url is going to redirect.
I know that when a url redirects to other url, the server sends an intermediate response containing a Location header field that tells the redirected url but I am not receiving that intermediate response through the url.openConnection(); method.

Simply call getUrl() on URLConnection instance after calling getInputStream():
URLConnection con = new URL( url ).openConnection();
System.out.println( "orignal url: " + con.getURL() );
con.connect();
System.out.println( "connected url: " + con.getURL() );
InputStream is = con.getInputStream();
System.out.println( "redirected url: " + con.getURL() );
is.close();
If you need to know whether the redirection happened before actually getting it's contents, here is the sample code:
HttpURLConnection con = (HttpURLConnection)(new URL( url ).openConnection());
con.setInstanceFollowRedirects( false );
con.connect();
int responseCode = con.getResponseCode();
System.out.println( responseCode );
String location = con.getHeaderField( "Location" );
System.out.println( location );

You need to cast the URLConnection to HttpURLConnection and instruct it to not follow the redirects by setting HttpURLConnection#setInstanceFollowRedirects() to false. You can also set it globally by HttpURLConnection#setFollowRedirects().
You only need to handle redirects yourself then. Check the response code by HttpURLConnection#getResponseCode(), grab the Location header by URLConnection#getHeaderField() and then fire a new HTTP request on it.

public static URL getFinalURL(URL url) {
try {
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setInstanceFollowRedirects(false);
con.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36");
con.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
con.addRequestProperty("Referer", "https://www.google.com/");
con.connect();
//con.getInputStream();
int resCode = con.getResponseCode();
if (resCode == HttpURLConnection.HTTP_SEE_OTHER
|| resCode == HttpURLConnection.HTTP_MOVED_PERM
|| resCode == HttpURLConnection.HTTP_MOVED_TEMP) {
String Location = con.getHeaderField("Location");
if (Location.startsWith("/")) {
Location = url.getProtocol() + "://" + url.getHost() + Location;
}
return getFinalURL(new URL(Location));
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
return url;
}
To get "User-Agent" and "Referer" by yourself, just go to developer mode of one of your installed browser (E.g. press F12 on Google Chrome). Then go to tab 'Network' and then click on one of the requests. You should see it's details. Just press 'Headers' sub tab (the image below)

Have a look at the HttpURLConnection class API documentation, especially setInstanceFollowRedirects().

I'd actually suggest using a solid open-source library as an http client. If you take a look at http client by ASF you'll find life a lot easier. It is an easy-to-use,scalable and robust client for http.

#balusC I did as you wrote . In my case , I've added cookie information to be able to reuse the session .
// get the cookie if need
String cookies = conn.getHeaderField("Set-Cookie");
// open the new connnection again
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setRequestProperty("Cookie", cookies);

java HttpsURLConnection

I tried to make this curl request executable from Java:
curl -H 'Accept: application/vnd.twitchtv.v2+json' \
-d "channel[status]=testing+some+stuff" \
-X PUT https://api.twitch.tv/kraken/channels/testacc222?oauth_token=6e7b9cyfi8zk1gr8g06eecebnitlcvb
My solution looks like this:
public static void main(String args[]) throws IOException {
String uri = "https://api.twitch.tv/kraken/channels/testacc222?oauth_token=6e7b9cyfi8zk1gr8g06eecebnitlcvb";
URL url = new URL(uri);
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setRequestMethod("PUT");
conn.setDoOutput(true);
conn.setRequestProperty("Accept", "application/vnd.twitchtv.v2+json");
String data = "channel[status]=testing";
OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
out.write(data);
out.flush();
for (Entry<String, List<String>> header : conn.getHeaderFields().entrySet()) {
System.out.println(header.getKey() + "=" + header.getValue());
}
}
I don't see any problem yet all it returns is:
Status=[400 Bad Request]
null=[HTTP/1.1 400 Bad Request]
Server=[nginx]
X-Request-Id=[ccc7a9a4a327b18ea4bf496f1f314fb8]
X-Runtime=[0.032328]
Connection=[keep-alive]
X-MH-Cache=[appcache1; M]
Date=[Sun, 06 Jul 2014 14:07:49 GMT]
Via=[1.1 varnish]
Accept-Ranges=[bytes]
X-Varnish=[2778442693]
X-UA-Compatible=[IE=Edge,chrome=1]
Cache-Control=[max-age=0, private, must-revalidate]
Vary=[Accept-Encoding]
Content-Length=[83]
Age=[0]
X-API-Version=[2]
Content-Type=[application/json; charset=utf-8]
I'm trying to figure this out for over a week now and I just don't see the mistake. Any help whatsoever would be greatly appreciated.

Try examining the response body, as it probably contains details about the rejection. Since the Content-Type specifies utf-8, you can create an InputStreamReader using that:
try (Reader response =
new InputStreamReader(conn.getErrorStream(), StandardCharsets.UTF_8)) {
int c;
while ((c = response.read()) >= 0) {
System.out.print((char) c);
}
}
Update: The response body states that the 'channel' parameter isn't present. This is because curl automatically encodes the POST data as application/x-www-form-urlencoded, but your code does not. You'll need to use URLEncoder on your data and also set the request's Content-Type:
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setRequestMethod("PUT");
conn.setDoOutput(true);
conn.setRequestProperty("Accept", "application/vnd.twitchtv.v2+json");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
String data = "channel[status]=testing";
data = URLEncoder.encode(data, "UTF-8");

FileNotFoundException with 404 status for valid URL on HTTP GET request

I have the following code to perform a GET request on the following URL:
http://rt.hnnnglmbrg.de/server.php/someReferenceNumber
However, here is my output from Logcat:
java.io.FileNotFoundException: http://rt.hnnnglmbrg.de/server.php/6
Why does it return 404 when the URL is clearly valid?
Here is my connect code:
/**
* Performs an HTTP GET request that returns base64 data from the server
*
* #param ref
* The Accident's reference
* #return The base64 data from the server.
*/
public static String performGet(String ref) {
String returnRef = null;
try {
URL url = new URL(SERVER_URL + "/" + ref);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuilder builder = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
returnRef = builder.toString();
} catch (IOException e) {
e.printStackTrace();
}
return returnRef;
}

When you request the URL, it actually return HTTP code 404 which mean not found. If you have control to the PHP script, set the header to 200 to indicate file is found.

You are getting a 404, as said above. To avoid an exception, try something like this:
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.connect () ;
int code = con.getResponseCode() ;
if (code == HttpURLConnection.HTTP_NOT_FOUND)
{
// Handle error
}
else
{
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
// etc...
}

Never trust what you see in your browser. Always try to mimic your request using something like curl, and you'll clearly see that you're getting an HTTP 404 response code.
java.net will translate the HTTP 404 code to a FileNotFoundException
curl -v http://rt.hnnnglmbrg.de/server.php/4
* About to connect() to rt.hnnnglmbrg.de port 80 (#0)
* Trying 217.160.115.112... connected
* Connected to rt.hnnnglmbrg.de (217.160.115.112) port 80 (#0)
> GET /server.php/4 HTTP/1.1
> User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
> Host: rt.hnnnglmbrg.de
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Mon, 11 Jun 2012 07:34:55 GMT
< Server: Apache
< X-Powered-By: PHP/5.2.17
< Transfer-Encoding: chunked
< Content-Type: text/html
<
* Connection #0 to host rt.hnnnglmbrg.de left intact
* Closing connection #0
0
From the javadocs at http://docs.oracle.com/javase/6/docs/api/java/net/HttpURLConnection.html
Returns the error stream if the connection failed but the server sent useful data nonetheless. The typical example is when an HTTP server responds with a 404, which will cause a FileNotFoundException to be thrown in connect, but the server sent an HTML help page with suggestions as to what to do.