Our project consists of 3 webapplcations that communicate with each other via web services.
All 3 web apps are running on 3 different web servers that run as a cluster with load balancer. (spring , tomcat, mysql)
Our CTO mentioned that in production, it can be very helpfull to invistigate errors on log on a single unified log file that is consist of all the webapplication log files combined together.
this way it is very easy to see in the log the whole flow across the webapps and not skipping from one log file to another (for each webapp log)
after a quick research we found that combining all the logs into a single file may cause corrupt file error of the log file itself. (we are using slf4j with log4j configuration)
So basically we have 3 questions:
1) Is it a good practice to combine all of the web apps log into one?
2) Whats the best way to achieve that (non corrupted log file will be nice)
3) Is it possible \ relevant to do the same concept of log unification in regard to tomcat logs? (unify all unified logs of all tomcats in the same cluster)
Logging to the same file from multiple servers can get very messy. You inevitably end up with multiple servers attempting to update files simultaneously, which has a habit of causing problems such as weirdly intermingled output and locking.
Given that you're using Log4J, then you should check out JMS queue appenders:
http://logging.apache.org/log4j/2.x/manual/appenders.html#JMSQueueAppender
Using this, every server logs to a JMS queue, and you can set up a listener which logs to file on a separate server.
A reasonable alternative would be to do a little bit of Perl scripting to grab the files and merge them periodically, or on demand.
You will probably find messages which are out of step with each other. That's because each server will be buffering up log output to avoid blocking your application processes.
Logging just the errors to a common place is useful. You can continue to log to each application's log, but you can add remote logging for selected entries (e.g. anything with Level=ERROR).
The easiest way to set this up is to run a SocketServer on one of your machines.
Here's an example that shows how to configure it.
Related
I have created a Spring Boot JMS application. It's function is to act as middleware which consumes/listens messages (xml) from a SOURCE system, transforms the xml, sends the transformed xml to a DESTINATION system.
I have already deployed the jar file in a Linux server. This is the first time I deployed an application, I am not sure of the correct way of keeping a history of log to a file should any error occur while the spring boot app is consuming and processing XML messages.
Some of the XML messages contain account numbers and if anything fails, I need to have some way of knowing which account failed.
I'm unsure because when working in the IDE, when you run the spring boot application, we normally see in the console a log of what is happening. But after I deployed the jar to the Linux server, I no longer have an IDE console to see what's happening. I just see the jar application running in port 8080.
In the IDE, normally we output messages using LOGGER.info(), LOGGER.error()...
private static final Logger LOGGER = LoggerFactory.getLogger(SomeClassFile.class);
What would be the best approach to keep history of logs?
Possible scenarios would be failure in connection while consuming messages from SOURCE system OR while sending messages to DESTINATION system.
Another possible failure would be, failure to transform XML messages.
All of that needs to be captured.
I deployed the app by creating a simple jenkins task which copies the jar to the Linux server after building.
I'd appreciate any comment or suggestion.
Thank you.
Reading the 12factor app in the logging chapter, it suggests that the application logs should be sent to STDOUT.
I found plenty of documentation on how to get the logs from STDOUT and send it to Cloud Watch when I'm running the application in a container.
But, is it possible (or even recommended) to do the same when running the application in an EC2 instance (no container/docker involved)?
The way that I managed to have my logs sent to Cloud Watch was doing what I would assume to be the standard way:
Configure my logback-spring.xml to log to a file (Java application)
Install the Cloud Watch agent on the instance and configure it to monitor the file above.
Happy life, all works.
I found this post on the AWS forum where it's suggested to create a symbolic link from stdout to a file, and I would assume that this file would have to be monitored by the agent. The benefit that I can see on this approach would be that who is developing the application don't need to worry about log config, just send to stdout and who is deploying the application could configure the way it wants using some script at the startup.
But as a drawback, I can't see a way to have the application's log sent to differents streams and/or groups.
Thank you.
I am building an application that plays opus audio files stored on a nas mounted on the application server. The application reads these files, and send them to the HTTP response output stream.
I am using spring-mvc and the web application runs on tomcat.
I get the audio files using:
<mvc:resources mapping="/audios/**" location="file:/path/to/the/mounted/nas"/>
and I serve them in a <audio> tag.
Now I would like to make the application extremely reactive and multi user friendly. If I have 20 people playing many audio simultaneously, the system gets very slow and I do not know what is the best thing to do.
Could you please suggest me how to improve the application
Thanks
You have to first investigate where is the actual delay happening. Is it in getting resource from file system from within the code, or it is because Tomcat is not able to withstand more load. To make sure it's not the code issue, you can log the code execution time and check if everything is fine there. Here is the way to do that - How do I time a method's execution in Java?
Now if it's the Tomcat server which is not able to withstand the load, which looks more probable issue, you might want to consider having cluster of multiple tomcat instances by creating a load balancer configuration with Apache web server. This way requests traffic load will be distributed by web server to all the running tomcat nodes, which will reduce the latency.
You can find how to configure load balancing at these link-
http://www.tutorialspoint.com/white-papers/load-balancing-and-scalability-via-tomcat-clusters.htm
https://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html
I made a web app with a ServletContextListener that in its contextInitialized() just calls two methods from two jars. The jars use log4j to print logs in its execution, what is useful for me on testing. Now, I package this app in an ear file, and deploy into a WAS, that is located into a server to which I don't have access (meaning, I can't access FTP to see paths on the server, etc) to set the path for the logs in the server, and it will take some time to get actual permission to set this.
I thought that maybe I could set log4j to print the logs in my working machine (that is also connected to the network the server is connected)
Is there anyway to do this?
Log4j has many appenders: http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Appender.html
For Log to different machine you can use SocketAppender. But for yours situation better store logs in database with JDBCAppender
I have developed a simple application such as insertion, updation in jsp,servlet that is java web. What i want to do is: suppose i am inserting data into my oracle db then data is inserted and i got a message that data has been inserted successfully, same thing i want to show in a command prompt when my web app is running in tomcat server. Application will run in browser and simultaneously command prompt will show that :
this data is inserted into this table---transaction committed successfully----
How can i implement that? Any help is much appreciated
The vast majority of Java web applications use Log4j for logging.
To log messages to the server's console, use the Log4j ConsoleAppender class.
Normally you need to do little more than drop log4j.jar into your tomcat/lib directory, and configure logging with a log4j.xml or log4j.properties file, then have your servlets or jsps or controllers or services issue logging commands. There are many tutorials on the web that can get you started learning log4j with tomcat.
Oh and by the way, you can try plain old System.out.println() calls from a servlet, though this is only for learning. Professional applications should use a logger.