Develop Reference

How does browser handle http requests

Hi I have some problems trying to handle http requests on my own and show them on screen using webview
Using the
shouldInterceptRequest
function in WebViewClient
So far, things are fine. I handle the matter well and receive cookies from the response headers and send them again in the next request.
But I'm having problems processing post requests
As the server expects some information from me in the request body, such as the password, email, and some other parameters that differ from one site to another.
Now my problem is how do I know what parameters are required from the server, such as the browser, as the browser handles these requests with ease and dwarfs sending all the required parameters
I tried to monitor requests via DevTools in PC chrome and I saw what parameters are required to login in only one site
I managed to get these parameters with javascript via
wb.loadUrl("javascript:const formData = new FormData(document.querySelector('form')); for(var pair of formData.entries()){INTER.putForms(pair[0],pair[1]); }");
I made a javainterface and put in it a putforms function that receives the information and puts it in a hashmap
Now I have solved the problem in one site, what about the rest of the sites? This method is not easy at all, I have to know how the browser deals with these requests
Please Help

Use httpclient to click next button?

I am writing a program to scrape the source code off a website. Each time the next button is clicked to go to the next page on the website a post request it sent.
I have been looking at using httpclient to take care of this issue, and have looked through examples and the httpclient API, but I cant seem to figure out whether httpclient can do this. Is this a function of httpclient, and if so what class would go about doing this?

I think that you're saying the webpage you're performing an http get on contains a "next button" on it, and that when you view the webpage in the browser and click the next button, the next page of the website is displayed.
If this is the case, yes, http client is able to do the same thing. But understand that http client does not integrate with your web browser. But you could scour the source code returned from the http get request using a library like jsoup to extract the url for the "next" page on the website, and then issue another http get to get that resource.
Assuming you already have the code for http client to issue the initial http get request, there is no additional api that is required. You just make another request after your program discovers the url for the "next" resource.

Dynamically update JSP page via Servlet

I am new to web programming with Java. I have a Client/Server written (in Java) and I want updates from the Client to be sent from the Server to a web interface for a user to view. The timing of the updates will be random, but I want to be able to dynamically update a web page with new data without the browser having to refresh.
Hope this makes sense:
I've tried creating a Servlet that Observes (implements Observer) my Server (which extends Observable) for updates (Strings), however I don't know how to dynamically add these to a browser window. I have tried printing directly from the Servlet using PrintWriter out = response.getWriter(); in the doGet() method, but response.getWriter() is unavailable in the Servlet's update() method.
My initial thoughts were to use a JSP page (I need to eventually incorporate HTML/CSS) that receives the Observer updates from the Server but I'm not sure.
I've done some research into Comet/PUSH, but I'm not sure if this is what I need - perhaps a bit overkill? Any advice on how to achieve what I'm after would be greatly appreciated.

The most common way to do this is for the client to poll the server for changes. Use an AJAX request on the client side to poll an endpoint on your server.
You will then need to use JQuery / Javascript to update your web page with the new data retrieved from the AJAX request.
I would suggest using JQuery in theb rowser and using the AJAX function: http://api.jquery.com/jQuery.ajax/
It allows you to do a callback after the request and in that call back you can update the content of your web page with the data retrieved from the request.

You cannot update a web page from server, not without recurring to polling or push.
If you want a true push, then comet is not overkill, otherwise you can use a polling script on the web page.

Loading resources before running java code in servlet

I have a servlet in tomcat. It takes a really long time for the java code in the backend to execute. Is there a way to load static resources (css,images,javascript) in parallel with the code in the backend? Right now, they are only loaded once the code finishes running.

You could use an Ajax-style solution where you paint your page without data, with a placeholder for retrieving the data, maybe even with a "loading" spinner graphic.
The way that an Ajax call works, when the page is loaded, some Javascript will fire that will launch an Ajax request to Tomcat via XmlHttpRequest that will start the calculation. The browser will notify the browser when the tomcat request is completed. Then there will be some javascript in the webpage that will take the response and replace the placeholder. If the server returns an HTML fragment, it's as simple as executing in javascript placeholder-div.innerHtml = your-response-text.
Here's a basic tutorial on Ajax and a Java-based example that has the web front-end communicating with a Java Servlet back-end.

How to handle authentication through AJAX with a java web app that uses form based login

I have a java web application running on WebSphere 7. The application uses form authentication method and part of the application accesses some protected resources using ajax requests. However, when the user's session expires, I am getting the login page in place of the content that is supposed to be refreshed by the ajax request.
Is there a good way to handle this problem? WebSphere returns a response status 200 with the login page so I cannot rely on that.
Maybe there is a way to tell the server that basic authentication should be used in certain circumstances but I don't know how.
I also thought of checking first if the session is new by making a request to unprotected resources first then return a certain status but it looks like a code smell solution...

This is how I handled it in a similar situation. In our case, the AJAX response is always JSON. When the login expires, the authentication filter always sends a login form in HTML. So I check the content-type like this,
if ((this.getHeader('Content-type') || '').include('application/json'))
If it's not JSON, I simply redirect to another protected page, which will trigger a full screen login and then that page will direct user back to the AJAX page.

You can send back some unique response or some error code(make sure you wont get this error code as valid response in any case) when the user session is not there to the Ajax call from WebSphere. And in the Ajax call method, on process response, check whether its error code.If its error code, redirect him to login page or do what ever and other case will be the valid data.