Develop Reference

Problems logging in to a webpage using java and jaunt-api

So I'm trying to log in to a webpage using Jaunt. The first thing to mention is that the webpage is .aspx and the submit button has an option onclick="javascript:WebForm_DoP..." and as far as I know Jaunt doesn't support Javascript right?
In case I'm wrong, the code I'm using is the one in the examples of Jaunt:
Form form = userAgent.doc.getForm(0);
form.setTextField("Login1$UserName","USER");
form.setPassword("Login1$Password","PASSWORD");
form.setCheckBox("Login1$RememberMe",false);
form.submit("GO");
System.out.println(userAgent.getLocation());
All the names and values are correct, and the user and password works since I can log in using the web browser. After I execute the code, in the output I get this:
message: UserAgent.sendPOST; Connection error requestUrl:
http://webpagehere.com/default.aspx [posting
__VIEWSTATE=%2FwEPDwUJLTk5MDc0NjQ2ZBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAgURTG9naW4xJFJlbWVtYmVyTWUFF0xvZ2luMSRMb2dpbkltYWdlQnV0dG9upWcarODJIwpeMt8HCmfaBn6iMWI%3D&__VIEWSTATEGENERATOR=CA0B0334&Login1%24UserName=USER&Login1%24Password=PASSWORD&Login1%24LoginButton=GO]
response: [none]
The form div is this one:
<form name="form1" method="post" action="Default.aspx" onsubmit="javascript:return WebForm_OnSubmit();" id="form1" style="text-align:center">
Any ideas what could be my problem? In case Jaunt doesn't allow me to do this login, could someone please recommend me a library for web scraping and interaction? Thanks!

Seems like you are stuck. Actually .aspx pages uses AJAX pagination. You will have to extract the values of __VIEWSTATE, __VIEWSTATEGENERATOR and all other form values and then send them with POST method in the request body. You can use Fiddler to get the request body which contains all these hidden variables and your entries to the form.
In Java you can use Selenium Or HTMLUnit which are Java GUI-Less browser, supporting JavaScript, to run agains web pages.
edit: You can use Jaunt-api as well, I just tried it with it, all you do is send a POST request alongwith the request-body, you can easily check it with Fiddler, and it works!!
Form values in HTTP POSTs are sent in the request body, in the same format as the querystring. You can find the request body of a link by inspecting it using the Fiddler and then copy request body from Textview and send the encoded data as request body.
UserAgent userAgent = new UserAgent();
userAgent.sendPOST("<your link to form page>","<request body>");

response.sendRedirect() from jsp:include being ignored?

I've got a jsp file, which includes another jsp file to check some values and such:
<jsp:include page="setup.jsp" />
Inside the setup.jsp I've got some conditional code which determines if some needed values are set in the session and if not redirects them to a different page. Or at least it is supposed to, but the redirect seems to be getting ignored.
System.err.println("Redirecting!");
response.sendRedirect("http://www.google.com");
return;
I see "Redirecting!" get logged to the console, but the page continues on and renders normally. I had curl dump the headers for me and saw that the response is HTTP/1.1 200 OK so it definitely isn't sending a 302 redirect.
Any idea what the problem is and how I can fix this?
EDIT: I have verified that my response is not yet committed. response.isCommitted() returns false meaning the status code and headers have not been sent yet.
EDIT 2: I've tried calling response.sendRedirect() in many other places and find that I can successfully redirect before the . The redirect inside the JSP seems to be ignored and if I try to redirect right AFTER the jsp then I get an illegal state exception because the response has already been committed.

The <jsp:include> uses under the covers RequestDispatcher#include(). Click the link to see the javadoc. Here's an extract of relevance (emphasis mine):
...
The ServletResponse object has its path elements and parameters remain unchanged from the caller's. The included servlet cannot change the response status code or set headers; any attempt to make a change is ignored.
...
The HttpServletResponse#sendRedirect() basically sets the HTTP response status to 302 and the HTTP Location header to the target URL. It is thus totally being ignored.
The deeper problem is that you're abusing JSP as a page controller/filter. You should actually be using a normal servlet or filter class for this which runs far before JSP.

The redirect header (I believe) needs to be at the top of the page. View the HTML spec for reference.
Have you tried placing it at the very top of the page? A code sample would help us debug...

Updating the url bar from the webapp to represent the current state

I would like to basically do what Jason asked for here
In one sentence, I would like the url bar to represent the state of the AJAX application so that I can allow to bookmark it as well as allow the user to return to the previous state by using the back/forward buttons in the browser.
The difference for me (From what Jason asked) is that I am using JSF 2.0.
I've read that JSF 2.0 added the ability to use get, but I am not sure what the correct way to use this.
Thanks for the help.
Further Clarification
If I understand correctly, to be able to bookmark specific states in the AJAX webapp I will have to use the location.hash. Am I correct? I'm trying to achieve a gmail-like behaviour in the sense that, while the app is complete AJAXified and no redirects occur, I can still use Back/Forward and bookmark (And that's why I would like the URL bar to be updated from the AJAX app itself and not through redirection)
Update
Just found this similar question

The difference for me (From what Jason asked) is that I am using JSF 2.0. I've read that JSF 2.0 added the ability to use get, but I am not sure what the correct way to use this.
Please note that this is not the same as maintaining the Ajax state. It usually happens by fragment identifiers (the part starting with # in URL, also known as hashbang). JSF doesn't offer builtin components/functionality for this. As far I have also not seen a component library which does that. You may however find this answer useful to get started with a homegrown hash fragment processor in JSF.
As to using GET requests, just use <h:link>, <h:outputLink> or even <a> to create GET links. You can supply request parameters in the h: components by <f:param>. E.g.
<h:link value="Edit product" outcome="product/edit">
<f:param name="id" value="#{product.id}" />
</h:link>
In the product/edit.xhtml page you can define parameters to set and actions to execute upon a GET request
<f:metadata>
<f:viewParam name="id" value="#{productEditor.id}" />
<f:event type="preRenderView" listener="#{productEditor.init}" />
</f:metadata>
In the request or view scoped bean associated with product/edit.xhtml page -in this example #{productEditor}-, you just define the properties and the listener method. The listener method will be executed after all properties are been gathered, converted, validated and updated in the model.
private Long id;
private Product product;
public void init() {
product = productService.find(id);
}

Normally you'd use AJAX to prevent complete page refreshes. AFAIK all current browsers would issue a page refresh if you change the base uri. Thus you would have to use the hash part as suggested in the question you provided.
We had a similar problem and did something like this:
We settled for the fact that users cannot bookmark the url.
For URLs that should be unique/bookmarkable we used different links that issue a redirect. Those URLs are provided in a sitemap.
For browser back, we added an intermediate page after login. This page does navigation and a redirect to the application. The navigation is stored in the session and when the server gets a navigation request (which can be a history back) the corresponding state is restored. A browser back opens that intermediate page which issues a redirect along with a navigation request on the server side.

uploading files by url

I need to implement a servlet that uploads files to a server, I realize everyone says it has to be a POST method in regard to uploading files and not with GET method. However is there a way to upload a file and have the parameters of the request show up in the url even if the request is coming from POST method? If not, is there another approach?
Currently my servlet using post method is http://example.com/FileUpload/UploadFile
What I want is somehting like http://example.com/FileUpload/UploadFile?id=125&fileNum=5
Thanks for your input.

Simply POST to
http://example.com/FileUpload/UploadFile?id=125&fileNum=5
instead of
http://example.com/FileUpload/UploadFile
There is no such restriction that you cannot post to an URL having parameters. You can process the post data as you are doing now, plus, you can get the get parameters also.

I think it would not be an elegant solution, but you could use JavaScript to alter the action of the form element before submitting it to include querystring parameters.
The form will be something like:
<form method="POST" id="myForm" onSubmit="submitMyForm(this)>
<input type="text" id="id">
Then you will need JavaScript to change the action element of the form:
function submitMyForm(theForm) {
theForm.action="http://example.com/FileUpload/UploadFile?id=" +
getElementById("id").value;
theForm.submit();
}
Is there some reason you cannot just submit the parameters with post and pull them out on the server side?

Alternatively, if you do a multipart/form-data post you can include multiple parameters along with your file. The parameters are sent as part of the post body, along with the file.

You can send parameters and files in the POST. For example in the html you can have a form with this values, they can be of hidden type.
In the servlet you can get the values in the same way you do using the GET.
It is also better to use the POST method because the user can't change the value in the URL direction bar.

webpage expired when back button pressed on JSF(Richfaces) submit

My title maybe confusing so please read on. I'm using the following technologies if you may. Spring, Hibernate, JSF (RichFaces), MySQL, Internet Explorer.
I have a List of items which is displayed in a RichFaces datatable like so:
item a
item b
item c
item d
item e
On the same page I have the following buttons: search, edit, add, delete and new.
When an user enters a search string, e.g. "item c", and press search button, then it displays a list of matching items, e.g:
item c
When the user presses the new button, the request will be redirected to another page using:
FacesContext facesContext = FacesContext.getCurrentInstance();
facesContext.getExternalContext().redirect(page + ".jsf" );
When the browser back button of IE is been pressed on that page, the page displays "web page expired". What is this and how can I avoid this?

web page expired
You will get this error when you're trying to obtain a non-cached POST request from the browser history. This behaviour is fully expected. To fix this "problem", you need to either turn the cache on or to replace POST by GET.
Enabling the browser cache is actually easy: just remove the Cache-Control: no-cache and related headers from the HTTP response of the POST request in question. The enduser will then only get a warning dialog that the POST data will be resent to the server, which in case of fully non-idempotent requests like placing an order or deleting an item is really not desirable. Replacing POST by GET is then a better solution. Getting searchresults (like as Google does) should really be done by GET.
Replacing POST by GET isn't easy in JSF prior to version 2.0. Best what you can do is to fire a redirect after the POST and pass the data of interest as request parameter which you retain from #{param} as managed property (more recommended) or store the data of interest in session scope (not recommended). A completely different alternative is to replace the JSF <h:form> by a simple HTML <form action="searchresults.jsf"> and do the search job in a #PostConstruct method in the backing bean associated with searchresults.jsf, after the submitted query has been gathered as managed property or from request parameter map.