I have created a jsp page with database connectivity. This page has both html content and java programming.My database consists of a list of ip addresses.
My java code fetches each ip address and checks whether it is currently alive on the network or not. So my jsp page loads only after this java code has performed checks on all ip addresses.This is why my page loads very late.
Is there any remedy to this so that my page loads quicker??
You can load all ip addresses from db into an ArrayList and also load all ips which are alive into another ArrayList and compare these two arrays. This should be much faster.
Separating JSP from Java code is one best practice, but the idea I'll describe here is more generally about separating the retrieval and updating of data from the rendering of the data, which is a common problem to solve.
What you need to do is separate the java code making all the network calls from the JSP which is being rendered. You can have the network calls all being run in one thread, checking each address once per minute or every few minutes, and updating each address' database record with a status. Then when the JSP is called, the JSP just grabs the latest data from the database and displays it (which is how JSP's should be used).
Now, there are numerous ways to accomplish this. If I were doing it myself, I would use Spring Framework and put the network-calling code in a method annotated with #Scheduled, and the network calls and database update could be done from that method. Details on how to use Spring are outside the scope of this answer, but hopefully this gives you an idea of the overall approach, and one technology you could start investigating.
I think there are two issues:
binding your JSP directly into your actual functionality. It would be preferable to implement some MVC structuring, and to allow the JSP to issue commands, and display whether those commands are being executed, if results are available etc. e.g. a command from the JSP to your servlet would initiate the processing (in a separate thread), and set state such that the JSP can report that 'processing' is in progress.
Your core functionality is to interrogate different IP addresses. That could easily be parallelised, such that you issue each IP query on a separate thread (naive solution, admittedly). Check out the Executor frameworks for more info.
-You should load JSP page only with IP list and after it's loaded, you can fetch IP adress statuses with AJAX requests.
-Earlier mentioned idea of caching statuses is a great.
-Also you can improve interface (paging, lazy loading lists, etc) to reduce count of IP addresses for checking.
Related
My employer has currently given me a project that has me scratching my head about synchronization.
I'm going to first talk about the situation I'm in:
I've been asked to create a pdf-report/quotation-tool that takes data (from csv-files; because the actual database the data is on is being used by old IBM software and they for reasons (unknown) don't want any direct access to this database (so instead of making copies of the data to other databases, they apparently found it incredibly fine to just create a folder on the server with loads and loads and loads of CSV-files.)), this piece of software is to load data into the application, query it, transform where needed, do calculations and then return with a pdf-file to the end-user.
The problem here is that getting, querying, and calculating things takes a fair amount of time, the other problem is: they want it to be a WebApp because the business team does not want to install any new software, they're mostly moving towards doing everything online (since the start of the pandemic), it being a WebApp means that every computation has to be done by the WebApp and getting the data likewise.
My question: Is each call to a servlet by a separate user treated as a separate servlet and should I only synchronize the methods on the business logic (getting and using the data); or should I write some code that puts itself in the middle of the servlet, receives a user-id (as reference), that then runs the business-logic in a synchronized-fashion, then receiving data and returning the pdf-file?
(I hope you get the gist of it...)
Everything will run on Apache Tomcat 8 if that helps. Build is Java 11lts.
Sorry, no code yet. But I've made some drawings.
With java web applications, the usual pattern is for the components to not have conversational state (meaning information specific to a specific user's request). If you need to keep state for a user on the server, you can use the http session. With a SPA or Ajax application it's often easier to keep a lot of that kind of state in the browser. The less state you keep on the server the easier things are as your application scales, you don't have to pin sessions to servers (messing up load balancing) or copy lots of session state across a cluster.
For simple (non-reactive) web apps that do blocking i/o, each request-response cycle gets its own dedicated thread from tomcat's pool. That thread delivers the http request to the servlet, handles the business logic and blocks while talking to the database, then carries the http response.
(Reactive webapps are going to be more complex to build, you will need a non-blocking database driver and you will have less choices for databases, so I would steer clear of those, at least for your first web application.)
The threadpool used by tomcat has to protect itself from concurrent access but that doesn't impact your code. Likewise there are 3rd party middletier caching libraries that have to deal with concurrency but you can avoid dealing with it directly. All of your logic is confined to one thread so it doesn't interfere with processing done by other threads unless there are shared mutable data structures. Those data structures would be the part of the application where synchronization might be one of several possible solutions.
Synchronization or other locking schemes are local to one instance of the application. If you want to stand up multiple instances of this application then you need to be aware each one would be locking separately from the others. So for some things it's better to do locking in the database, since that is shared across webapp instances.
If you can make use of a database to store your data, so that you can rely on the database for caching and indexing, then it seems likely your application should be able to avoid having doing a lot of locking.
If you want examples there are a lot of small examples for building web apps using spring at https://spring.io/guides. These are spring boot applications that are self hosted so you can put them together quickly and run them right away.
Going rogue with a database may not be the best course since databases need looking after by DBAs. My advice is put together two project plans, one for using a database, and one for using the flat files. The flat file one will have to allow for addressing issues like handling caching, indexing data, replication of data from the legacy database, and not having standard tools that generate pdfs from sql queries. The alternative plan using a database should have a lot less sorting out of infrastructure and a shorter time til you can get down to cranking out reports.
I have a Spring MVC project in Java. This web app can be accessed by multiple users in different browsers. I haven't coded any session bean in my program.
Now I want to 'crash'/'timeout' the browsing of one of the users, while other users will go on with their normal expected browsing. I want to do this to see if this action has any effect on the shared variables.
What kind of coding I need to do for this? Thanks in advance!
It is not at all clear what you are trying to achieve here, but I'm assuming that you are doing this as an experiment ... to see what happens.
You could modify the webapp to implement some special request, or request parameter, or request parameter value that tells the webapp to crash or freeze the request being processed. Then send that request from one browser while others are doing "normal" things.
Whether this is going to reveal anything interesting is ... questionable.
Another interpretation is that you are aiming to include timed out requests and other things in your normal testing regime. To achieve that, you would need implement some kind of test harness to automate the sending of requests to your server; i.e. to simulate a number of simultaneous users doing things. There are various test tools for doing that kind of thing.
I'm working on a site containing real estate listings in Spring MVC. I would like to prevent scripts to steal the content by scraping the site. Does anyone have experience with techniques that can easily be plugged in to a spring mvc environment?
User-agent is too simple to circumvent.
One idea I had was to keep track of two counters on the serverside.
ipaddress --> (counter xhr request, counter page request)
the counter page request is increased with a filter
the counter xhr request is increased on document ready
If a filter notices the two counters are totally out of sync, the ip is blocked.
Could this work or are there easier techniques?
Cheers
edit
I am aware that if scrapers are persistent they will find a way to get the content. However, I'd like to make it as hard as possible.
Off the top of my head:
Look for patterns in how your pages are requested. Regular intervals is a flag. Regular frequency might be a flag (four times a day, but at different times during the day).
Require login. Nothing gets shown until the user logs in, so at least the scraper has to have an account.
Mix up the tag names around the content every once in a while. It might break their script. Do this enough times and they'll search for greener pastures.
You can't stop it at all, but you can make it harder as much as possible.
One way to make it harder is change your content URL very frequent base on time with appending some encrypted flag in url.
Some of suggestion are in given link.
http://blog.screen-scraper.com/2009/08/17/further-thoughts-on-hindering-screen-scraping/
http://www.hyperarts.com/blog/the-definitive-guide-to-blog-content-scraping-how-to-stop-it/
Load the content via ajax.
Make the ajax request dynamic so they cant just go and scrape the ajax request.
Only sophisticated scrapers support execution of java script.
Most scrapers dont run the pages through a real browser, so you can try to use that to your advantage.
I need to check if existing web application is ready the be deployed in a clustered environment.
Cluster:
Several Linux boxes. The flow is controlled by a load balancer that is using simple round robin algorithm with sticky session.
Application
Stateless (hopefully) java web application that retrieves content from back office and format it appropriately.
I have access to the source code. What should I check in the code to be sure that it will run in the cluster?
Check that something is not cached in a memory or file system that stores state of the application.
...Something else?
If you're using EJBs (which is recommended if you access a DB), then here is a list of restrictions:
http://java.sun.com/blueprints/qanda/ejb_tier/restrictions.html
I guess similar restrictions apply to the web application.
The easiest way to check the application is to start by having it running on 2 servers with the same data so at startup both are in the same state. Let's assume for a user to complete an operation, the browser will make 2 consecutive HTTP requests to your web app -- what you need to do is hit webserver 1 with first call and web server 2 with second call; then try the other way around, then with both requests going to the same webserver -- and if you get the same result each time you're very likely you have ready-to-cluster application. (It doesn't mean the app IS ready to cluster as there might be object states etc it stores in memory which are not easy to spot from the front-end, but it gives you a higher probability that IT MIGHT BE ok to run in a cluster.)
If its truly "stateless", there would be no problem, you could make any request of any server at any time and everything would just work. Most things aren't quite that easy so any sort of state would either have to be streamed to and from the page as it moves from client to server, or be stored on the back end, and have some sort of token passed back and forth in order to retrieve it from whatever shared data store you're using for that. If they are using the HttpSession, then anything that is retrieved from the session, if modified, needs to be set back into the session with session.setAttribute(key,value). This setting the attribute acts as a signal that whatever is being stored in the session needs to be replicated to the redundant servers. Make sure anything stored in the session implements, and actually is, Serializable. Some servers will allow you to store objects, (I'm looking at you weblogic), but will then throw an exception when it tries to replicate the object. I've had many a coworker complain that having to set stuff back to the session should be redundant, and perhaps it should, but this is just the way things work.
Having state is not a big problem if done properly. Anyway, all applications have state. Even if serving somewhat static file, the file content associated with an URL is indeed part of the state.
The problem is how this state is propagated and shared.
state inside user session is a no brainer. Use a session replication mechanism (slower but no session loss on node crash) or session sticky load balancer and your problem is solved.
All other shared state is indeed a problem. In particular even cache state must be shared and perfectly coherent otherwise a refresh on the same page could generate different result on random depending on witch web server, and thus the cache you hit.
You can still cache data using a shared cached (like ehcache), or failing back to session sticky.
I guess it is pretty difficult to be sure that the application will indeed work in a clusterised environement because a singleton in some obscure service, a static member somewhere, anything can potentially produce strange results. You can validate the general architecture for sure, but you'll need to do in reality and perform some validation test before going into production.
I'm using Java+struts2+JSP as the web application framework.
I have to pass some huge objects through struts actions to my jsp pages. this makes the pages so heavy to load and on the other hand they suck the server's bandwidth out.
Is there any way to send compressed objects via struts2 to a jsp page and decompress them there?
The question is a bit vague on how the objects are passed from the action classes to the JSP pages, but it appears to me that instead of forwarding the request during the execution of the request, the application is issuing a client-side re-direct to a new page.
In the JSP/servlet model, forwards are internal to the server, and do not result in a new request by the client. On the other hand, redirects will result in the browser being forced to go the new page as indicated by the server.
If possible, you should investigate the use of forwards which is the default mechanism in Struts to display the view. This will only reduce the server's bandwidth requirements.
On the topic of the large memory consumption in JSP pages, you might want to profile the application to deduce whether the 'huge' load time of JSPs is due these objects or whether it is due to the additional client request as explained above. Without such a profile report indicating CPU and memory usage, it is presumptuous to claim that object bloat is responsible for high page load times.
If you need to move data inside your server side, check this:
http://www.google.de/search?q=java+gzip&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a
If you want to improve download speed for clients, enable gzip compression in your webserver.
Sounds like you need to unzip files with JavaScript. This Answer actually provides a link to just such JavaScript. I don't know how practical the idea is though.