How to track my projects like let's say i have 5 different projects which are already in production and i want to know which user in a single day how many times login to my application out of 5 application and after login how many times he clicked on a button or how many times he/she click on a function provided by the application. Now i want to track all these things without doing maximum changes in existing code?
Only way I can think of is through logging, using log4j or something similar. Also, maybe checkout Splunk.
If it is JavaEE web application, You can use
Java Servlet Filter
for auditing where information about operating requests and the outcome of those requests can be collected, stored, and distributed.Initially you can store these data in database and later on you can analyze it using different application.
Related
I need to run a very long process in a java based spring boot web application. The process consists of following steps.
Get details for about 3,00,000 users from the database.
Iterate over them.
Generate PDF file for each user using itext.
Save PDF file on the filesystem.
Update the database that the PDF file for the given user has been created.
Update the PDF path for the user in the database.
Now, this entire process can take lots of time. May be lots of hours or even may be days as it consist of creating pdf file for each user, then lots of db updates.
Also, I need this process to run in background so that the rest of the web application can run smoothly.
I am thinking of using Spring Batch or Messaging Queue. Haven't really used any of them, so not sure if they are proper frameworks for such kind of problem or which one of these two is best fit for the problem.
What is the ideal way to implement such kind of tasks?
If you can't name a requirement you expect to be satisfied by a framework / library you most likely won't need one...
Generating PDFs might need a lot of power, you might want to keep this background process away from your main web application on it's own machines.
If it's a simple java process it's usually easier to control and to move it around your environment.
To me this looks like a simple task for "plain" java - KISS. Or am I missing something?
I'd make sure the Finder used to fetch the users from the database is
restartable, i.e. only fetches unprocessed users (in case you have to stop the processing because shit happens:-)
runs in batches to keep the db round trips and load low
is multi threadable i.e. can fetch users split into a given number of threads (userid mod numberOfThreads, assuming userId is evenly distributed) so you can add more machines / threads if necessary.
You should use spring batch for this process. When the user presses the button, you would launch the job asynchronously. It will then run in a separate thread and process all your records. The current status of the job can be obtained from the job repository. Spring batch is made for this type of processing.
Could someone explain how the web apps work on play framework example? The things I misunderstand:
1)Which part of the code(I mean code from examples listed on play framework site) allows many users to use the same app at same time?
2)For example I have a program : a textField, button and list. Write in textfield, press button and the text is added to the list. Every user should have own list, but where should I store the data if it's objects, in memory? In DB? In session/cookies? And how this data should be recognized, by session of? Are there any good examples?
I think you should read the documentation and search the web a bit more about Play to be honest. Here are a few pointers:-
Play will use a default thread pool to serve requests. All actions are asynchronous - meaning (as long as you do not block a thread in the action) a Play app should be able to serve a large volume of requests (depending on your machine resources), as these threads get quickly reused to serve other requests
Play is designed to be stateless so no Session (in context of Java servlet session that is). Play does not use Java servlet spec. (actually one of the original drivers Play was created was of limitations of Servlet spec)
In terms of your own application, what are you going to do with that data you have collected later? What do you need it for? That should help you determine where to store it (if at all). If you want it just for session scope then you could use session cookie - see the docs). If you want it later (ie. when user comes back to site) then you could put it in a store of some kind. Again, see the docs.
There are loads of articles on Play's architecture. And the official docs are quite good at pointing you in the right direction for most use cases.
The Lightbend has some good resources and there are loads of Activator templates to provide codes samples.
Here is the simple CRUD template which might help you.
What is the best way to keep a log of user changes in my web application (java/tomcat/struts/mysql)? I give out accounts and each account has multiple users. I want the account administrators to be able to see who did what at any given time. And I'd like to be able to access ALL of it. First, I need a way to know which fields have been changed, then I need to log the changes for each account in a place where they can see them. Obviously, I don't want to slow the app down. I read an answer on this site suggesting keeping a db log - querying the database for changes after each query is sent. Wasn't sure how to do that.
This depends on the nature of your web application. Let's assume your web application is a e-commerce system and it allows the user to add new product, or updating an existing product. When a user perform a specific action like adding a new product, the basic goal is to capture his user name, action and time stamp. Same for updating a product, you might want to keep track what values he updated, what was the old value and when did he change that.
To achieve this, firstly you need to
Create an audit table
Obviously you want to keep track the last modified person, timestamp, created by and etc.
Create a logging mechanism whenever some changes/actions performed.
There are few ways to do this, you can either do it via application or leave everything to database trigger. I would suggest to use triggers to detect any Create/Update/Delete event in the database, and ask the trigger to capture the details and write to the Audit table. I think this is the cleanest and less maintenance way. However, if you want to log using application, you have to make code changes, create new methods to capture the details to the Audit table in your action classes.
More information on MYSQL Trigger here
I was looking on a similar "Method" to log the transactions and other stuffs in my web app. Just while browsing Google, i found this link:
https://www.owasp.org/index.php/Logging_Cheat_Sheet telling about two possible ways to log: Either on database or on filesystem at some log files...
When using the file system, it is preferable to use a separate
partition than those used by the operating system, other application
files and user generated content For file-based logs, apply strict
permissions concerning which users can access the directories, and the
permissions of files within the directories In web applications, the
logs should not be exposed in web-accessible locations, and if done
so, should have restricted access and be configured with a plain text
MIME type (not HTML) When using a database, it is preferable to
utilize a separate database account that is only used for writing log
data and which has very restrictive database , table, function and
command permissions Use standard formats over secure protocols to
record and send event data, or log files, to other systems e.g. Common
Log File System (CLFS), Common Event Format (CEF) over syslog,
possibly Common Event Expression (CEE) in future; standard formats
facilitate integration with centralised logging services
They've beautifully explained the possible ways we can log, what should be logged, what to be avoided too.
Hope it's useful to you.
I am trying to build an e-commerce java web application, I am running into an issue for booking/purchasing items from the store. Each item can only be purchased once, my issue is that users on two different machines might be purchasing the same item, and for that I was thinking of synchronization. I don't quite understand synchronization and how the application can still run when no one is using it. Does the application run on server side? Can someone please give me some information on this?
One of the possible ways is to lock the field in the database. Locking in the database will prevent it to be used from any application that tries to access it. If you use lock, "synchronization" , from the server side app, you must be sure that only that app have access to the database or you could sell the same product twice. Some study about your database lock mechanism maybe can help you to prevent this issue.
what kind of logging frame work or API to use for swing applications which is used by multiple users in Unix.
Is it possible to log all verbose/exception in one file per day or event one user one file per day? Since the user can open the same application with multiple instance.
I also have another solution is to save the exceptions into database. But if I miss the excetpions, those will not be saved in DB.
anybody has better solutions? Thank you very much!
You might like this article and discussion. The author mentions java.util.logging, which is discussed more extensively in this Java Logging Overview. In the context you describe, FileHandler should be able to sort out multiple instances per user without requiring a database.
If you are distributing your software across a network then you have less chances of knowing each and every event user does. Not sure If log4j or any other framework helps to track every user actions in your situation. Unless if you have something running on your app server.
Well..If I were you I will do it this way.
For exceptional conditions:
Come up good solid exceptional framework(something like assigning a unique Id for each exception).
In case of exception condition catch it and write the full stack trace to database table with the same unique id.
Come up some kind of search tool (web application) which helps you see what went wrong during user actions.
For Normal tracking I probably save user actions into table, but it hurts performance unless you come up with good framework. Not sure If I answered your questions. Please let me know if you have something to say.
-padur
Saving to database seems a good idea, something like when user logs in to your swing app. Create a file in user temp directory write all his actions/exceptions etc etc into the file and when he log out read the file and save it into database.Wells there are several ways to track user actions, this is one among them.