Need to write JUnit Test case

Need to write JUnit Test case - java

I am a fresher java developer.
But I dont have much knowledge about writing Junit test cases.
I am going to have a job test soon.
For which they want me to write a program
To read HTML from any website say "http://www.google.com" ( You
can use any API of inbuilt APIs in Java like URLConnection )
Print on console the HTML from the url above and save it to a file (
web-content.txt) in local machine.
JUnit test cases for above
programme.
I have completed the first two steps as below:
import java.io.*;
import java.net.*;
public class JavaSourceViewer{
public static void main (String[] args) throws IOException{
System.out.print("Enter url of local for viewing html source code: ");
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String url = br.readLine();
try {
URL u = new URL(url);
HttpURLConnection uc = (HttpURLConnection) u.openConnection();
int code = uc.getResponseCode();
String response = uc.getResponseMessage();
System.out.println("HTTP/1.x " + code + " " + response);
InputStream in = new BufferedInputStream(uc.getInputStream());
Reader r = new InputStreamReader(in);
int c;
FileOutputStream fout=new FileOutputStream("D://web-content.txt");
while((c = r.read()) != -1){
System.out.print((char)c);
fout.write(c);
}
fout.close();
} catch(MalformedURLException ex) {
System.err.println(url + " is not a valid URL.");
} catch (IOException ie) {
System.out.println("Input/Output Error: " + ie.getMessage());
}
}
}
Now I need help with 3rd step.

You need to extract a method therefore like this
package abc.def;
import java.io.*;
import java.net.*;
public class JavaSourceViewer {
public static void main(String[] args) throws IOException {
System.out.print("Enter url of local for viewing html source code: ");
BufferedReader br =
new BufferedReader(new InputStreamReader(System.in));
String url = br.readLine();
try {
FileOutputStream fout = new FileOutputStream("D://web-content.txt");
writeURL2Stream(url, fout);
fout.close();
} catch (MalformedURLException ex) {
System.err.println(url + " is not a valid URL.");
} catch (IOException ie) {
System.out.println("Input/Output Error: " + ie.getMessage());
}
}
private static void writeURL2Stream(String url, OutputStream fout)
throws MalformedURLException, IOException {
URL u = new URL(url);
HttpURLConnection uc = (HttpURLConnection) u.openConnection();
int code = uc.getResponseCode();
String response = uc.getResponseMessage();
System.out.println("HTTP/1.x " + code + " " + response);
InputStream in = new BufferedInputStream(uc.getInputStream());
Reader r = new InputStreamReader(in);
int c;
while ((c = r.read()) != -1) {
System.out.print((char) c);
fout.write(c);
}
}
}
Ok, now you are able to write the JUnit-Test.
package abc.def;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.MalformedURLException;
import org.junit.Test;
import junit.framework.TestCase;
public class MainTestCase extends TestCase {
#Test
public static void test() throws MalformedURLException, IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
JavaSourceViewer.writeURL2Stream("http://www.google.de", baos);
assertTrue(baos.toString().contains("google"));
}
}

First move your code from the main method to other methods in the
JavaSourceViewer.
Second make these methods return something that
you can test. e.g. the file name of the output file or the Reader.
Then create a class for the unit test
import org.junit.* ;
import static org.junit.Assert.* ;
public class JavaSourceViewerTest {
#Test
public void testJavaSourceViewer() {
String url = "...";
JavaSourceViewer jsv = new JavaSourceViewer();
// call you methods here to parse the site
jsv.xxxMethod(...)
....
// call you checks here like:
// <file name to save to output data from your code, actual filename> - e.g.
assertEquals(jsv.getOutputFile(), "D://web-content.txt");
....
}
}
To execute the Junit use an IDE (like eclipse) or put the junit.jar file in the classpath and from the console:
java org.junit.runner.JUnitCore JavaSourceViewerTest

Your steps are not right. Doing it in this way would definitely help, 3, then 1, and then 2. Doing it this way will force you to think in terms of functionalities and units. And the result code will be testable, without doing anything special. Test first also guide the design of your system, besides providing you a safety net.
P.S. Never try to write the code before the test. It's simply not natural neither it gives much value, as you can see.
Now, to test the 1st unit, you can compare the string, the html from google.com, with some existing string. But that test case will break if Google changes it's page. Other way, is to just check the HTTP code from the header, if that's 200, you are fine. Just an idea.
For the second one, you can compare the string, you read from the web page, to string you wrote to the file, by reading the file.

String str = "";
URL oracle = new URL("http://www.google.com/");
BufferedReader in = new BufferedReader(
new InputStreamReader(oracle.openStream()));
File file = new File("C:/Users/rohit/Desktop/rr1.txt");
String inputLine;
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
bw.write(inputLine);
}
bw.close();
in.close();

Related

Scanner (Getinputstream()) throws exception returns no line found from URL connection

I am trying to write a Java program, which establishes connection to yahoo finance, and pulls some data of the website for a specific stock.
The program terminates with the exception no line found, which is thrown at the if(input.hasNextLine()) statement. I get what the exception mean, but i Can't figure out what the error is.
I know that the problem is not in the URL construction, because the URL downloads the requested data from the web, when copied into a web browser.
hopes someone can point me in the right direction, i have been puzzled for several hours, trying to search the forum, but no luck so far.
My code looks as follows:
import java.net.URL;
import java.net.URLConnection;
import java.util.Calendar;
import java.util.GregorianCalendar;
import java.util.Scanner;
public class Download {
public Download(String symbol, GregorianCalendar end, GregorianCalendar start){
//Creates the URL
String url = "http://chart.finance.yahoo.com/table.csv?s="+symbol+
"&a="+start.get(Calendar.MONTH)+
"&b="+start.get(Calendar.DAY_OF_MONTH)+
"&c="+start.get(Calendar.YEAR)+
"&d="+end.get(Calendar.MONTH)+
"&e="+end.get(Calendar.DAY_OF_MONTH)+
"&f="+end.get(Calendar.YEAR)+
"&g=d&ignore=.csv";
try{
//Creates the URL object, and establishes connection
URL yhoofin = new URL(url);
URLConnection data = yhoofin.openConnection();
//Opens an input stream, to read from
Scanner input = new Scanner(data.getInputStream(),"UTF-8");
System.out.println(input.nextLine());
//skips the first line,
if(input.hasNextLine()){
input.nextLine();
//tries to print the data.
while(input.hasNextLine()){
String line = input.nextLine();
System.out.println(line);
}
}
//closes connection
input.close();
}
catch(Exception e){
System.err.println(e);
}
}
}
with the following main method:
import java.util.GregorianCalendar;
public class test {
public static void main(String[] args){
GregorianCalendar start = new GregorianCalendar(2015,7,10);
GregorianCalendar end = new GregorianCalendar(2016,7,10);
String symbol ="NVO";
Download test = new Download(symbol,end,start);
System.out.println("Done");
}
}

//http://chart.finance.yahoo.com/table.csv?s=ABCB4.SA&a=1&b=19&c=2017&d=2&e=19&f=2017&g=d&ignore=.csv
String url = "http://chart.finance.yahoo.com/table.csv?s="+symbol+".SA"+
"&a="+start.get(Calendar.MONTH)+
"&b="+start.get(Calendar.DAY_OF_MONTH)+
"&c="+start.get(Calendar.YEAR)+
"&d="+end.get(Calendar.MONTH)+
"&e="+end.get(Calendar.DAY_OF_MONTH)+
"&f="+end.get(Calendar.YEAR)+
"&g=d&ignore=.csv";
System.out.println(url);
try
{
URL yhoofin = new URL(url);
URLConnection data = yhoofin.openConnection();
data.connect();//not necessary
System.out.println("Connection Open! = "+data.getHeaderFields().toString());
String redirect = data.getHeaderField("Location");
if (redirect != null){
data = new URL(redirect).openConnection();
}
BufferedReader in = new BufferedReader(new InputStreamReader(data.getInputStream()));
String inputLine;
System.out.println();
in.readLine();//skip first line
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
lines.add(inputLine);
}

Java - writing to url not working

I'm trying to make a java program that changes a text document on my website. The permissions are on that everyone can edit it. I've tried, and reading it works perfectly, but writing doesn't.
Here's the code for the writing:
import java.net.*;
import java.io.*;
public class Main {
public static void main(String[] args) throws Exception {
URL infoThing = new URL("http://www.[name of my website]/infoThing.txt");
URLConnection con = infoThing.openConnection();
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream());
out.write("Change to this.");
out.close();
}
}

For a Java program to interact with a server-side process it simply must be able to write to a URL, thus providing data to the server. It can do this by following these steps:
1 Create a URL.
2 Retrieve the URLConnection object.
3 Set output capability on the URLConnection.
4 Open a connection to the resource.
5 Get an output stream from the connection.
6 Write to the output stream.
7 Close the output stream.
If you want to write to the url. You have to use the concepts above and concepts for servlets.
An example program that runs the backwards script over the network through a URLConnection:
import java.io.*;
import java.net.*;
public class ReverseTest {
public static void main(String[] args) {
try {
if (args.length != 1) {
System.err.println("Usage: java ReverseTest string_to_reverse");
System.exit(1);
}
String stringToReverse = URLEncoder.encode(args[0]);
URL url = new URL("http://java.sun.com/cgi-bin/backwards");
URLConnection connection = url.openConnection();
PrintStream outStream = new PrintStream(connection.getOutputStream());
outStream.println("string=" + stringToReverse);
outStream.close();
DataInputStream inStream = new DataInputStream(connection.getInputStream());
String inputLine;
while ((inputLine = inStream.readLine()) != null) {
System.out.println(inputLine);
}
inStream.close();
} catch (MalformedURLException me) {
System.err.println("MalformedURLException: " + me);
} catch (IOException ioe) {
System.err.println("IOException: " + ioe);
}
}
}

eclipse juno showing error editor does not contain main file, i think the editor has some error not the code

import java.io.*;
import java.net.*;
public class JavaSourceViewer{
public static void main(String[] args) throws IOException {
String java.io.BufferedReader.readLine() throws IOException
System.out.print("Enter url of local for viewing html source code: ");
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String url = br.readLine();
try {
URL u = new URL(url);
HttpURLConnection uc = (HttpURLConnection) u.openConnection();
int code = uc.getResponseCode();
String response = uc.getResponseMessage();
System.out.println("HTTP/1.x " + code + " " + response);
InputStream in = new BufferedInputStream(uc.getInputStream());
Reader r = new InputStreamReader(in);
int c;
FileOutputStream fout=new FileOutputStream("D://web-content.txt");
while ((c = r.read()) != -1) {
System.out.print((char)c);
fout.write(c);
}
fout.close();
} catch (MalformedURLException ex) {
System.err.println(url + " is not a valid URL.");
} catch (IOException ie) {
System.out.println("Input/Output Error: " + ie.getMessage());
}
}
}
I dont know what is wrong with this, but the code is not running just showing the error.

It is the code. Looks like you've got a copy/paste error. Method declarations are not permitted within the body of other methods. Remove this partial method declaration from the main method
String java.io.BufferedReader.readLine() throws IOException

Sometimes restarting the eclipse solves it. Also make sure the jdk path is accurate and try to configure it again and finally clean the project using the eclipse tool

WGET download happens faster when executed through command line and slowly when executed through Java code

I am downloading a file using WGET through a java code, which takes around 10 mins to download 20 MB file. But on executing the wget download through command line, the same file gets downloaded in 7 seconds at 10MbPs speed. Does anyone know why this is? How can I improve my Java code?
Below is the code I have used to download a file using WGET. It takes around 10 minutes to download a 20 MB file. But when I run the wget command through the command line, it happens in seconds!!
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
public class WGETServer
{
public File download(URL sourceurl, String username, String password, String fileName)
{
//System.out.println("WGET download() is starting ...");
File file = null;
URLConnection urlConnection = null;
BufferedReader reader = null;
FileOutputStream outputStream = null;
try {
urlConnection = sourceurl.openConnection();
String userNameAndPassword = username +":"+ password;
String encoding = new sun.misc.BASE64Encoder().encode (userNameAndPassword.getBytes());
//The line which is supposed to add authorization data
urlConnection.setRequestProperty ("Authorization", "Basic " + encoding);
reader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
}
catch (IOException e) {
System.err.println("Internet connection failure or invalid Username/Password.");
return null;
}
try {
file = new File("file path");
outputStream = new FileOutputStream(file);
int character;
while((character = reader.read()) != -1)
{
outputStream.write(character);
}
outputStream.flush();
outputStream.close();
reader.close();
} catch (IOException e) {
System.err.println(e.getMessage());
return null;
}
System.out.println("downloading completed");
return file;
}
public static void main(String args[]) throws MalformedURLException
{
URL sourceurl = new URL("https:blablabla");
String username = "username";
String password = "password";
String filename = "filename";
WGETServer WGETdownload = new WGETServer();
WGETdownload.download(sourceurl, username, password, filename);
}
}

Wrap the FileOutputStream with a BufferedOutputStream.
new BufferedOutputStream(new FileOutputStream(...))
Otherwise each and every character written is synchronized to disk by the underlying operating system which is a time consuming process. This is why buffering is so important.

You have buffered reader (Good) but then you write the content char by char to the disk (BAD). That kills your performance. It's not the reading, it's the writing.

How do you Programmatically Download a Webpage in Java

I would like to be able to fetch a web page's html and save it to a String, so I can do some processing on it. Also, how could I handle various types of compression.
How would I go about doing that using Java?

I'd use a decent HTML parser like Jsoup. It's then as easy as:
String html = Jsoup.connect("http://stackoverflow.com").get().html();
It handles GZIP and chunked responses and character encoding fully transparently. It offers more advantages as well, like HTML traversing and manipulation by CSS selectors like as jQuery can do. You only have to grab it as Document, not as a String.
Document document = Jsoup.connect("http://google.com").get();
You really don't want to run basic String methods or even regex on HTML to process it.
See also:
What are the pros and cons of leading HTML parsers in Java?

Here's some tested code using Java's URL class. I'd recommend do a better job than I do here of handling the exceptions or passing them up the call stack, though.
public static void main(String[] args) {
URL url;
InputStream is = null;
BufferedReader br;
String line;
try {
url = new URL("http://stackoverflow.com/");
is = url.openStream(); // throws an IOException
br = new BufferedReader(new InputStreamReader(is));
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
try {
if (is != null) is.close();
} catch (IOException ioe) {
// nothing to see here
}
}
}

Bill's answer is very good, but you may want to do some things with the request like compression or user-agents. The following code shows how you can various types of compression to your requests.
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection(); // Cast shouldn't fail
HttpURLConnection.setFollowRedirects(true);
// allow both GZip and Deflate (ZLib) encodings
conn.setRequestProperty("Accept-Encoding", "gzip, deflate");
String encoding = conn.getContentEncoding();
InputStream inStr = null;
// create the appropriate stream wrapper based on
// the encoding type
if (encoding != null && encoding.equalsIgnoreCase("gzip")) {
inStr = new GZIPInputStream(conn.getInputStream());
} else if (encoding != null && encoding.equalsIgnoreCase("deflate")) {
inStr = new InflaterInputStream(conn.getInputStream(),
new Inflater(true));
} else {
inStr = conn.getInputStream();
}
To also set the user-agent add the following code:
conn.setRequestProperty ( "User-agent", "my agent name");

Well, you could go with the built-in libraries such as URL and URLConnection, but they don't give very much control.
Personally I'd go with the Apache HTTPClient library.
Edit: HTTPClient has been set to end of life by Apache. The replacement is: HTTP Components

All the above mentioned approaches do not download the web page text as it looks in the browser. these days a lot of data is loaded into browsers through scripts in html pages. none of above mentioned techniques supports scripts, they just downloads the html text only. HTMLUNIT supports the javascripts. so if you are looking to download the web page text as it looks in the browser then you should use HTMLUNIT.

You'd most likely need to extract code from a secure web page (https protocol). In the following example, the html file is being saved into c:\temp\filename.html Enjoy!
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
/**
* <b>Get the Html source from the secure url </b>
*/
public class HttpsClientUtil {
public static void main(String[] args) throws Exception {
String httpsURL = "https://stackoverflow.com";
String FILENAME = "c:\\temp\\filename.html";
BufferedWriter bw = new BufferedWriter(new FileWriter(FILENAME));
URL myurl = new URL(httpsURL);
HttpsURLConnection con = (HttpsURLConnection) myurl.openConnection();
con.setRequestProperty ( "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" );
InputStream ins = con.getInputStream();
InputStreamReader isr = new InputStreamReader(ins, "Windows-1252");
BufferedReader in = new BufferedReader(isr);
String inputLine;
// Write each line into the file
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
bw.write(inputLine);
}
in.close();
bw.close();
}
}

To do so using NIO.2 powerful Files.copy(InputStream in, Path target):
URL url = new URL( "http://download.me/" );
Files.copy( url.openStream(), Paths.get("downloaded.html" ) );

On a Unix/Linux box you could just run 'wget' but this is not really an option if you're writing a cross-platform client. Of course this assumes that you don't really want to do much with the data you download between the point of downloading it and it hitting the disk.

Get help from this class it get code and filter some information.
public class MainActivity extends AppCompatActivity {
EditText url;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate( savedInstanceState );
setContentView( R.layout.activity_main );
url = ((EditText)findViewById( R.id.editText));
DownloadCode obj = new DownloadCode();
try {
String des=" ";
String tag1= "<div class=\"description\">";
String l = obj.execute( "http://www.nu.edu.pk/Campus/Chiniot-Faisalabad/Faculty" ).get();
url.setText( l );
url.setText( " " );
String[] t1 = l.split(tag1);
String[] t2 = t1[0].split( "</div>" );
url.setText( t2[0] );
}
catch (Exception e)
{
Toast.makeText( this,e.toString(),Toast.LENGTH_SHORT ).show();
}
}
// input, extrafunctionrunparallel, output
class DownloadCode extends AsyncTask<String,Void,String>
{
#Override
protected String doInBackground(String... WebAddress) // string of webAddress separate by ','
{
String htmlcontent = " ";
try {
URL url = new URL( WebAddress[0] );
HttpURLConnection c = (HttpURLConnection) url.openConnection();
c.connect();
InputStream input = c.getInputStream();
int data;
InputStreamReader reader = new InputStreamReader( input );
data = reader.read();
while (data != -1)
{
char content = (char) data;
htmlcontent+=content;
data = reader.read();
}
}
catch (Exception e)
{
Log.i("Status : ",e.toString());
}
return htmlcontent;
}
}
}

Jetty has an HTTP client which can be use to download a web page.
package com.zetcode;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.api.ContentResponse;
public class ReadWebPageEx5 {
public static void main(String[] args) throws Exception {
HttpClient client = null;
try {
client = new HttpClient();
client.start();
String url = "http://example.com";
ContentResponse res = client.GET(url);
System.out.println(res.getContentAsString());
} finally {
if (client != null) {
client.stop();
}
}
}
}
The example prints the contents of a simple web page.
In a Reading a web page in Java tutorial I have written six examples of dowloading a web page programmaticaly in Java using URL, JSoup, HtmlCleaner, Apache HttpClient, Jetty HttpClient, and HtmlUnit.

I used the actual answer to this post (url) and writing the output into a
file.
package test;
import java.net.*;
import java.io.*;
public class PDFTest {
public static void main(String[] args) throws Exception {
try {
URL oracle = new URL("http://www.fetagracollege.org");
BufferedReader in = new BufferedReader(new InputStreamReader(oracle.openStream()));
String fileName = "D:\\a_01\\output.txt";
PrintWriter writer = new PrintWriter(fileName, "UTF-8");
OutputStream outputStream = new FileOutputStream(fileName);
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
writer.println(inputLine);
}
in.close();
} catch(Exception e) {
}
}
}

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

Need to write JUnit Test case - java

Related

Scanner (Getinputstream()) throws exception returns no line found from URL connection

Java - writing to url not working

eclipse juno showing error editor does not contain main file, i think the editor has some error not the code

WGET download happens faster when executed through command line and slowly when executed through Java code

How do you Programmatically Download a Webpage in Java

Categories

Resources