htmlunit can not call post ajax requests - java

I have a webpage that fills a datagrid with post ajax requests like this
$.ajax({
type: "POST",
url: "http://bss.bimser.com.tr/Handlers/eBADataGridHandler.ashx",
data: '{"ID":"dgSearchTickets","Type":0,"Page":"2","FilterInput":[],"QuickFilterText":null,"SortedColumn":""}',
success: function(msg){
console.log(msg);
}
});
When i try this with the below code i cant get html result; my code;
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
UnexpectedPage upage = webClient.getPage(webRequestPost);
HtmlPage page = HTMLParser.parseHtml(upage.getWebResponse(), webClient.getCurrentWindow());
WebRequest webRequestPost2 = new WebRequest(new URL(".../DataGridHandler.ashx"), HttpMethod.POST);
webRequestPost2.setAdditionalHeader("Content-Type", "application/x-www-form-urlencoded");
webRequestPost2.setRequestBody(DefaultValues.searchPagingJson2);
page = webClient.getPage(webRequestPost2);
this code returns an error webpage. How can i get html info ?
EDIT:
page.executeJavaScript("var request = new XMLHttpRequest();\n" +
"request.open('POST', 'http://bss.bimser.com.tr/Handlers/eBADataGridHandler.ashx', true);\n" +
"request.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');\n" +
"request.onload = function() {\n" +
" document.getElementsByTagName(\"body\")[0].innerHTML = request.responseText;\n" +
" console.log(request.responseText);\n" +
"};\n" +
"request.send('{\"ID\":\"dgSearchTickets\",\"Type\":0,\"Page\":\"%s\",\"FilterInput\":[],\"QuickFilterText\":null,\"SortedColumn\":\"Sent Date\"}');\n");
I was able to solve the problem by executing the code above. Now the problem is without manual waiting htmlunit does not wait for the result. I tried this below
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_52);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setAjaxController(new AjaxController() {
#Override
public boolean processSynchron(HtmlPage page, WebRequest request, boolean async) {
return true;
}
});

In general HtmlUnit can do ajax requests.
Creating you own web request is not the design idea of HtmlUnit. Think about HtmlUnit more like a browser automated by your code. Have a look at the samples on the HtmlUnit homepage.
If you really like to create your own requests you are responsible for the whole request. This includes various headers for security or session information. Install a web proxy like Chares and watch the communication of the real page. Then you might be able to create a similar request. And have a look at the returning webpage maybe there is a hint for you (like the login is missing).

Related

Handle resource with ajax call

My app suppose to download file from DB.
Controller:
#GetMapping(value = "/download/{id}")
public ResponseEntity<Resource> downloadBook(#PathVariable Long id) {
Book book = bookService.findById(id);
return ResponseEntity.ok().contentType(MediaType.APPLICATION_PDF)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + book.getFileName() + ".pdf\"")
.body(new ByteArrayResource(book.getFile()));
}
This works when i call it directly from browser (download pop up), but what i'm tring to do is handle it with ajax call. This is my ajax method so far, but this code actually just alert me with success with no dowload pop up.
downloadBook : function(bookId) {
$.ajax({
url : basePath + '/book/download/' + bookId,
success : function() {
alert("success!");
},
error : function() {
alert("error!");
}
});
}
See Download a file by jQuery.Ajax
What's happening here is that your ajax request is being made to /book/download/ as specified in the url: parameter. When the request completes successfully it fires the success callback. However, all you are doing there is showing an alert - with alert("success!") - so nothing further will happen.
The linked answer explains why you cannot simply do this with ajax like you might imagine. I haven't reiterated all of the linked answer as it's already been explained there, but the principal of that answer is exactly the same for your problem.
downloadBook : function(bookId) {
window.location = basePath + '/book/download/' + bookId
}
My downloadBook method looks like this now and it solved my problem.

How to "get around" steam login while using HtmlUnit?

I want to write a java program which can browse webpage associated with steam. The problem is that I don't understand how steam automatically logs in users.
I figured these services use some kind of a cookies, but i couldn't find those cookies in chrome cookie manager. So even if I write a program which clicks the login button, then goes to the login page there fills it out, the secure page comes up which asks for email confirmation, which I can't and don't want to program.
These sites are tf2outpost.com and backpack.tf.
Actually, there is an error with the website.
Go to http://www.tf2outpost.com/ with real browser
Click on 'Sign in through STEAM'
Check the source of the login page, at line #51 there is:
<script language="javascript">
function Logout()
{
$('actionInput').value = 'steam_openid_logout';
$('openidForm').submit();
}
>$J( function() { $J('#steamAccountName').focus() } );
</script>
The issue with >$J, which is incorrect JavaScript and is not tolerated by HtmlUnit.
To get around this, you can change the response as hinted here.
So you can have something like:
try (WebClient webClient = new WebClient(BrowserVersion.CHROME)) {
new WebConnectionWrapper(webClient) {
public WebResponse getResponse(WebRequest request) throws IOException {
WebResponse response = super.getResponse(request);
if (request.getUrl().toExternalForm().endsWith("identifier_select")) {
String content = response.getContentAsString("UTF-8");
content = content.replace(">$J( function()", "$J( function()");
WebResponseData data = new WebResponseData(content.getBytes("UTF-8"),
response.getStatusCode(), response.getStatusMessage(), response.getResponseHeaders());
response = new WebResponse(data, request, response.getLoadTime());
}
return response;
}
};
HtmlPage page = webClient.getPage("http://www.tf2outpost.com");
HtmlAnchor a = page.getAnchorByHref("/login");
page = a.click();
page.<HtmlInput>getHtmlElementById("steamAccountName").setValueAttribute(username);
page.<HtmlInput>getHtmlElementById("steamPassword").setValueAttribute(password);
page = page.<HtmlInput>getHtmlElementById("imageLogin").click();
webClient.waitForBackgroundJavaScript(5000);
System.out.println(page.asXml());
Still, here is captcha that came up.
Hope that helps.

Spring Security CSRF Token not working with AJAX

I have a problem in my spring boot app with the csrf token.
I have a form where I can edit a Person. A Person can have
Let us now imagine that the person has a car and enter this and store it. The next time he wants to delete this car and enter another one. I have created that so that there is a list of all of his cars -- he has the option to remove this from the list. Now I'm starting from these pills and want to send with the corresponding ID to the server a POST. When I try I get a 403 forbidden and I have no idea why.
If I change from POST to GET, then it works.
My JavaScript (taken from this site: http://docs.spring.io/autorepo/docs/spring-security/4.0.0.CI-SNAPSHOT/reference/htmlsingle/#the-csrfmetatags-tag)
var csrfParameter = $("meta[name='_csrf_parameter']").attr("content");
var csrfHeader = $("meta[name='_csrf_header']").attr("content");
var csrfToken = $("meta[name='_csrf']").attr("content");
// using JQuery to send a non-x-www-form-urlencoded request
var headers = {};
headers[csrfHeader] = csrfToken;
$.ajax({
url: "./delete/car",
type: "GET",
headers: headers,
});
$.ajax({
url: "./delete/car",
type: "POST",
headers: headers,
});
My controller methods:
#RequestMapping(value = "/{login}/delete/car", method = RequestMethod.GET)
public ModelAndView delete(#PathVariable("login") final String login) {
System.out.println("Stop");
return new ModelAndView("redirect:" + WebSecurityConfig.URL_PERSONS_OVERVIEW);
}
#RequestMapping(value = "/{login}/delete/car", method = RequestMethod.POST)
public ModelAndView deleteInstEmp(#PathVariable("login") final String login) {
System.out.println("Stop");
return new ModelAndView("redirect:" + WebSecurityConfig.URL_PERSONS_OVERVIEW);
}
Any suggestions?
Thanks in advance.
OK, after strugglin with all that, I get the following result.
I added the fail method to the Ajax construct and get the following message:
"Failed to execute 'setRequestHeader' on 'XMLHttpRequest': '${_csrf.headerName}' is not a valid HTTP header field name."
the official spring site advises that you have to put this: <sec:csrfMetaTags /> or from other sources, this: <meta name="_csrf" th:content="${_csrf.token}"/> in your html file.
After this, you should be able to access these attributes in your JavaScript, but in my case I get undefined and ${_csrf.headerName}.
A last try was to take the value from the hidden value (chapter 24.5: http://docs.spring.io/autorepo/docs/spring-security/4.0.0.CI-SNAPSHOT/reference/htmlsingle/#the-csrfmetatags-tag).
Now, I have the following:
$(function () {
var token = $("input[name='_csrf']").val();
var header = "X-CSRF-TOKEN";
$(document).ajaxSend(function(e, xhr, options) {
xhr.setRequestHeader(header, token);
});
});
$.ajax({
url: "./delete/car",
type: "POST",
success:function(response) {
alert(response);
}
});
With this it works like a charm.
Another way, you can use the following code:
$.ajax({
url : './delete/car',
headers: {"X-CSRF-TOKEN": $("input[name='_csrf']").val()},
type : 'POST',
success : function(result) {
alert(result.msgDetail);
}
})
I suggest you first check if a valid csrf token and the header have been generated using chrome debugger. If not, then have you added the <sec:csrfMetaTags /> in the <head>?(you will need to import the spring security taglibs). If using Apache tiles, you will have to add this at the <head> section of the template file being used for the view.
If the token is not empty, then in your security-context/configuration file, check if you have disabled csrf security by any chance. By default it is enabled and needs to be for this process to work.

AJAX variables in URL do not appear after XMLHttpRequest.open()

I'm making an application in Java EE and want to use Ajax to update something. Only the parameters are not appearing in the URL.
The good URL:
localhost:8080/Weblog/AddComment?commenttext=example&postid=3
This is what appears in browserbar as the URL:
localhost:8080/Weblog/AddComment
Javascript:
function doAddComment() {
var url = "AddComment?commenttext="+newcommentcontent.value+"&postid="+postid.value;
var req = getXHR();
req.onreadystatechange = function()
{ processRequestChange(req);
req.open("GET", url, true);
req.send(null);
}
}
What am I doing wrong?
You are making the request with Ajax, not sending the browser to a new page. The browser bar shows the URL of the current page, not the URL of the last HTTP resource requested.
Use the history API if you want to manipulate the address bar to have a history you can go back and forward through when fetching data via Ajax.
I had to change the GET to a POST.
function doAddComment() {
var url = "AddComment?commenttext="+newcommentcontent.value+"&postid="+postid.value;
var req = getXHR();
req.onreadystatechange = function()
{ processRequestChange(req);
req.open("POST", url, true);
req.send(null);
}
}
And now it works.

Simple AJAX support for Wicket 1.5.4

I'm new to wicket and AJAX and trying to set up a simple page that passes messages from the html page via jQuery & ajax to a wicket servlet. The page then updates a label with an appropriate response.
I tried to use the code below from Marrying Wicket And jQuery UI Autocomplete Ajax but the code does not compile using Wicket 1.5.4. getParameterMap(), setRequestTarget and StringRequestTarget are all unrecognised in wicket 1.5.4.
Any help would be much appreciated.
add(aab = new AbstractAjaxBehavior() {
#Override
public void renderHead(Component component, IHeaderResponse response) {
super.renderHead(component, response);
response.renderJavaScript("var callbackUrl = '" + aab1.getCallbackUrl() + "';", "callbackurl");
}
// handle the ajax request
#Override
public void onRequest() {
System.out.println("ajax request received");
RequestCycle requestCycle = RequestCycle.get();
Request request = requestCycle.getRequest();
IRequestParameters irp = request.getRequestParameters();
String json = getJSON();
requestCycle.scheduleRequestHandlerAfterCurrent(new TextRequestHandler("application/json", "UTF-8", json));
}
});
This was solved as the tutorial code was updated.

Categories

Resources