I have a situation where I have a series of similar JSPs, each of which is called from a servlet based upon an option entered by a user.
However, I would like to adjust these JSPs so that they can be additionally called in batch from a program which runs hourly on the server, and write the JSP output to text file.
Can anyone tell me how this might be done at all?
I am thinking along the lines of:
URL url = new java.net.URL("http://127.0.0.1/myServlet");
URLConnection con = url.openConnection();
Or is there a better way?
OK: I must be doing something very foolish here because this doesn't appear to work: I have a batch program which runs every hour and it contains the following code:
try {
URL url = new java.net.URL("http://127.0.0.1:8084//myApp//myServletMapping?par=parValue");
URLConnection connection = url.openConnection();
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setDoInput(true);
InputStream response = connection.getInputStream();
}
catch (Exception ex) {
logger.error("Error calling servlet in batch", ex);
}
According to my understanding of the instructions in this tutorial, the above should be enough to trigger the get method in the servlet which is mapped to by myServletMapping in the code above. This servlet's get method contains a simple System.out.println("Here"); which I would expect to see.
What am I doing wrong?
Or is there a better way?
Not really. That's about as basic as it gets. The servlet is "called" when the server receives an HTTP, and that's exactly what your proposed code will do.
You could use a library such as HTTPUnit, or a different programming language, but it's all going to boil down to sending an HTTP request.
In addition to what Matt said about the servlet being called when you make a HTTP request:
There is no need to a Java class that is called from the batch file. Just use wget to retrieve the page:
wget http://127.0.0.1/myServlet -O mypage.jsp
wget is open source and available for (nearly) all operating systems
I think your solution is fine. The only thing I would change is not actually do the reading of the resource your self. I would suggest using Google's Resources.toString lib like so
String data = Resources.toString(url, ...);
im not sure what you mean by asking
I would like to adjust these JSPs so that they can be additionally called in batch
do they need authentication ?
anyways, depending on your requirements you could even try it with spring batch
http://static.springsource.org/spring-batch/
you could also look at Apache HTTP Client library... again this might be too much of a library to call a simple Servlet..
If possible you should refactor your servlets and move the logic out to a service class that doesn't rely on request/response so that you can call this from wherever you like.
I did something similar two weeks ago - a report that was generated from the web app needed to be ran as a scheduled job. I moved all the code from a Spring controller (what was it doing there anyway?) into a service class and called that code from the controller and a scheduled Quartz job.
Related
I have a few questions about a specific REST call I'm making in JAVA. I'm quite the novice, so I've cobbled this together from several sources. The call itself looks like this:
String src = AaRestCall.subTrackingNum(trackingNum);
The Rest call class looks like this:
public class AaRestCall {
public static String subTrackingNum (Sting trackingNum) throws IOException {
URL url = new URL("https://.../rest/" + trackingNum);
String query = "{'TRACKINGNUM': trackingNum}";
//make connection
URLConnection urlc = url.openConnection();
//use post mode
urlc.setDoOutput(true);
urlc.setAllowUserInteraction(false);
//send query
PrintStream ps = new PrintStream(urlc.getOutputStream());
ps.print(query);
ps.close();
//get result
BufferedReader br = new BufferedReader(new InputStreamReader(urlc
.getInputStream()));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line=br.readLine())!=null) {
sb.append(line);
}
br.close();
return sb.toString();
}
}
Now, I have a few questions on top of the what is wrong with this in general.
1) If this rest call is returning a JSON object, is that going to get screwed up by going to a String?
2) What's the best way to parse out the JSON that is returning?
3) I'm not really certain how to format the query field. I assume that's supposed to be documented in the REST API?
Thanks in advance.
REST is a pattern applied on top of HTTP. From your questions, it seems to me that you first need to understand how HTTP (and chatty socket protocols in general) works and what the Java API offers for deal with it.
You can use whatever Json library out there to parse the HTTP response body (provided it's a 200 OK, that you need to check for, and also watch out for HTTP redirects!), but it's not how things are usually built.
If the service exposes a real RESTful interface (opposed to a simpler HTTP+JSON) you'll need to use four HTTP verbs, and URLConnection doesn't let you do so. Plus, you'll likely want to add headers for authentication, or maybe cookies (which in fact are just HTTP headers, but are still worth to be considered separately). So my suggestion is building the client-side part of the service with the HttpClient from Apache commons, or maybe some JAX-RS library with client support (for example Apache CXF). In that way you'll have full control of the communication while also getting nicer abstractions to work with, instead of consuming the InputStream provided by your URLConnection and manually serializing/deserializing parameters/responses.
Regarding the bit about how to format the query field, again you first need to grasp the basics of HTTP. Anyway, the definite answer depends on the remote service implementation, but you'll face four options:
The query string in the service URL
A form-encoded body of your HTTP request
A multipart body of your HTTP request (similar to the former, but the different MIME type is enough to give some headache) - this is often used in HTTP+JSON services that also have a website, and the same URL can be used for uploading a form that contains a file input
A service-defined (for example application/json, or application/xml) encoding for your HTTP body (again, it's really the same as the previous two points, but the different MIME encoding means that you'll have to use a different API)
Oh my. There are a couple of areas where you can improve on this code. I'm not even going to point out the errors since I'd like you to replace the HTTP calls with a HTTP client library. I'm also unaware of the spec required by your API so getting you to use the POST or GET methods properly at this level of abstraction will take more work.
1) If this rest call is returning a JSON object, is that going to get
screwed up by going to a String?
No, but marshalling that json into an obect is your job. A library like google gson can help.
2) What's the best way to parse out the JSON that is returning?
I like to use gson like I mentioned above, but you can use another marshal/unmarhal library.
3) I'm not really certain how to format the query field. I assume
that's supposed to be documented in the REST API?
Yes. Take a look at the documentation and come up with java objects that mirror the json structure. You can then parse them with the following code.
gson.fromJson(json, MyStructure.class);
Http client
Please take a look at writing your HTTP client using a library like apache HTTP client which will make your job much easier.
Testing
Since you seem to be new to this, I'd also suggest you take a look at a tool like Postman which can help you test your API calls if you suspect that the code you've written is faulty.
I think that you should use a REST client library instead of writing your own, unless it is for educational purposes - then by all means go nuts!
The REST service will respond to your call with a HTTP response, the payload may and may not be formatted as a JSON string. If it is, I suggest that you use a JSON parsing library to convert that String into a Java representation.
And yes, you will have to resort to the particular REST API:s documentation for details.
P.S. The java URL class is broken, use URI instead.
I am trying to simulate HTTP requests in Java with the URL class and the HttpURLConnection class, the GET requests are easy to simulate while some POST requests seem harder because most POST request need Cookie in the request header. Some cookies were set by the HTTP response in the Set-Cookie field and I can get them by the function provided by HttpURLConnection, but I found that other cookies may be set by JavaScript and I have no way to handle them, so I wonder is there any packaged tool to simulate HTTP requests in Java?
try Apache commons Httpclient:
http://hc.apache.org/httpclient-3.x/
Why do you need to generate HTTP requests? Do you want to perform some stress tests?
I'd advise using something like JMeter (you can find a brief tutorial here).
Hope this helps something, it's better to avoid reinventing the wheel (if you need something like this, but it wasn't clear for me from your question).
For the cookie set with Javascript, you could try to parse the HTTP response, extract the cookie information and set it for your next request
For example, lets say, the response has code which calls a setCookie() function (setCookie is user-defined javascript function),
...
//some javascript code
setCookie("username", "johndoe");
//some more javascript
...
then you would extract the line setCookie() and the parse it for the name and value
I am trying to read a website using the java.net package classes. The site has content, and i see it manually in html source utilities in the browser. When I get its response code and try to view the site using java, it connects successfully but interprets the site as one without content(204 code). What is going on and is it possible to get around this to view the html automatically.
thanks for your responses:
Do you need the URL?
here is the code:
URL hef=new URL(the website);
BufferedReader kj=null;
int kjkj=((HttpURLConnection)hef.openConnection()).getResponseCode();
System.out.println(kjkj);
String j=((HttpURLConnection)hef.openConnection()).getResponseMessage();
System.out.println(j);
URLConnection g=hef.openConnection();
g.connect();
try{
kj=new BufferedReader(new InputStreamReader(g.getInputStream()));
while(kj.readLine()!=null)
{
String y=kj.readLine();
System.out.println(y);
}
}
finally
{
if(kj!=null)
{
kj.close();
}
}
}
Suggestions:
Assert than when manually accessing the site (with a web browser client) you are effectively getting a 200 return code
Make sure that the HTTP request issued from the automated (java-based) logic is similar/identical to that of what is sent by an interactive web browser client. In particular, make sure the User-Agent is identical (some sites purposely alter their responses depending on the agent).
You can use a packet sniffer, maybe something like Fiddler2 to see exactly what is being sent and received to/from the server
I'm not sure that the java.net package is robot-aware, but that could be a factor as well (can you check if the underlying site has robot.txt files).
Edit:
assuming you are using the java.net package's HttpURLConnection class, the "robot" hypothesis doesn't apply.
On the other hand you'll probably want to use the connection's setRequestProperty() method to prepare the desired HTTP header for the request (so they match these from the web browser client)
Maybe you can post the relevant portions of your code.
I have abandoned my earlier quest to make the applet communicate directly with the database, even though users and webpages have said that it's possible. I am now trying to get my applet to pass information (String and boolean format) entered in textfields or indicated by checkboxes, and give this to the servlet, which then stores it appropriately in the database. I've got the applet front end - the GUI - built, and the servlet - database connection also built. The only problem is the link between the two, applet and servlet. How would one pass String data from an applet to a servlet?
Thanks,
Joseph G.
First up, you have to acknowledge that you can only communicate with the server from where your applet was downloaded from, that includes the port number, unless you want to mess around with permissions, applet signing and all that malarky. This also isn't just an Applet restriction, the same applies to Flash and JavaScript (though in the case of JavaScript there are tricks to get around it).
Using either the "getCodeBase()" or "getDocumentBase()" method on your Applet will get you a URL from which you can get the component parts required to build a new URL that will let you call a servlet.
Thus, your Applet must be being served from the same server that your servlet is hosted on.
e.g. if your Applet is in the following page:
http://www.example.com/myapplet.html
...it means you can make calls to any URL that starts with
http://www.example.com/
...relatively easily.
The following is a crude, untested, example showing how to call a Servlet. This assumes that this snippet of code is being called from within an instance of Applet.
URL codeBase = getCodeBase();
URL servletURL = new URL(codeBase.getProtocol(), codeBase.getHost(), codeBase.getPort(), "/myServlet");
// assumes protocol is http, could be https
HttpURLConnection conn = (HttpURLConnection)servletURL.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
PrintWriter out = new PrintWriter(conn.openOutputStream());
out.println("hello world");
out.close();
System.out.println(conn.getResponseCode());
Then in your servlet, you can get the text sent by overriding doPost() and reading the input stream from the request (no exception handling shown and only reads first line of input):
public void doPost(HttpServletRequest req, HttpServletResponse res) {
BufferedReader reader = req.getReader();
String line = reader.readLine();
System.out.println("servlet received text: " + line);
}
Of course, that's just one approach. You could also take your inputs and build up a query string like this (URLEncoding not shown):
String queryString = "inputa=" + view.getInputA() + "&inputb=" + view.getInputB();
and append that to your URL:
URL servletURL = new URL(codeBase.getProtocol(), codeBase.getHost(), codeBase.getPort(), "/myServlet?" + queryString);
However, it seems fairly common to build up some kind of string and stream it to the servlet instead these days.
A recommended format would be JSON as it's semi-structured, while being easy to read and there are plenty of (de)serializers around that should work in your Applet and in your servlet. This means you can have a nice object model for your data which you could share between your Applet and Servlet. Building up a query string of complex inputs can be a mind bender.
Likewise, you could actually use Java serialisation and stream binary to your Servlet which then uses Java serialisation to create the appropriate Java objects. However, if you stick to something like JSON, it'll mean your servlet is more open to re-use since Java serialisation has never been implemented outside of Java (that I am aware of).
Hm, I guess the applet and the servlet run in two separate Java processes. In that case you'll have to use some remoting technology, e.g. an http call to localhost. In fact, that is what servlets are mainly used and implemented for: Accept and process http requests.
I'm attempting to build a Tropo Ruby application and I need to retrieve the result of an HTTPS GET. The Tropo platform doesn't have the httpclient Ruby gem so I can't use that. The Ruby engine used is JRuby so a suggestion has been to make use of the Java URL class to do the request. I've played around with it a little bit and I seem to be able to create the URL object ok but am now struggling with how to get the results of executing the request. How do I do it?
javaURL = java.net.URL.new svcURL
transferResult = javaURL.getContent()
I am not sure what getContent() does exactly, but you can use openStream() which gives you an InputStream to read from.
javaURL = java.net.URL.new svcURL
transfer = javaURL.openStream()
// read from stream
transfer.close()