I am using log4j-core-2.5 for my logging and I would like to Discard items when the logging queue is full. I have determined that this could be applied by using the log4j2.asyncQueueFullPolicy configuration property in log4j.
Could somone please help me with how to apply this configuration in my application.
I am using a file configuration like so...
Configurator.initialize("", "log4j-node.xml");
My config file looks somthing like this
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<ThresholdFilter level="DEBUG" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="[%d{dd-MMM HH:mm:ss.SSS} %logger{1}.%t %-5level] %msg%n"/>
</Console>
<RollingFile name="DefaultFile" fileName="../logs/app.log"
filePattern="../logs/app-%d{MM-dd-yyyy}-%i.log" bufferedIO="true" bufferedSize="32000" immediateFlush="false">
<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
...
</RollingFile>
<Async name="AsyncFile">
<AppenderRef ref="DefaultFile"/>
<LinkedTransferQueue/>
</Async>
<Async name="AsyncConsole">
<AppenderRef ref="Console"/>
<LinkedTransferQueue/>
</Async>
</Appenders>
<logger name="org.hibernate">
<level value="debug" />
</logger>
<Loggers>
<Root level="TRACE">
<AppenderRef ref="AsyncConsole"/>
<AppenderRef ref="AsyncFile"/>
</Root>
<Logger name="org.hibernate.SQL" level="error" />
<!-- Log all JDBC parameters -->
<Logger name="org.hibernate.type" level="error" />
<!-- Log all SQL DDL statements as they are executed -->
<Logger name="org.hibernate.tool.hbm2ddl" level="error" />
<!-- Log the state of all entities (max 20 entities) associated with the session at flush time -->
<Logger name="org.hibernate.pretty" level="error" />
<!-- Log all second-level cache activity -->
<Logger name="org.hibernate.cache" level="error" />
<!-- Log transaction related activity -->
<Logger name="org.hibernate.transaction" level="error" />
<!-- Log all JDBC resource acquisition -->
<Logger name="org.hibernate.jdbc" level="error" />
<!-- Log HQL and SQL ASTs during query parsing -->
<Logger name="org.hibernate.hql.ast.AST" level="error" />
<!-- Log all JAAS authorization requests -->
<Logger name="org.hibernate.secure" level="error" />
<!-- Log everything (a lot of information, but very useful for troubleshooting) -->
<Logger name="org.hibernate" level="error" />
</Loggers>
</Configuration>
Cheers
First, please upgrade to Log4j 2.10, which fixed an important issue in this area where log events no longer appear out of order in the log file when the queue is full.
You can configure Log4j2's behaviour via system property log4j2.AsyncQueueFullPolicy. See AsyncQueueFullPolicy and AsyncQueueFullPolicyFactory. The "Discard" policy with some log level may be of interest.
Property log4j2.AsyncQueueFullPolicy controls the routing
behaviour. If this property is not specified or has value "Default",
this factory creates DefaultAsyncQueueFullPolicy objects.
If this property has value "Discard", this factory creates
DiscardingAsyncQueueFullPolicy objects. By default, this router
discards events of level INFO, DEBUG and TRACE if the queue is full.
This can be adjusted with property log4j2.DiscardThreshold (name of
the level at which to start discarding).
Related
I am currently working on tidying up the logs for an internal tool we have. I came across the use of ${sys:logger.out.type} inside the Loggers in log4j2.xml.
Here's a copy of our log4j2.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="OFF">
<Appenders>
<Console name="stdout" target="SYSTEM_OUT">..
<PatternLayout pattern="..."/>
</Console>
<RollingFile name="FILE" fileName="${sys:microservice.log}" filePattern="${sys:microservice.log}.%i">
<PatternLayout>
<Pattern>...</Pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy />
<SizeBasedTriggeringPolicy size="300 MB" />
</Policies>
<DefaultRolloverStrategy max="5" />
</RollingFile>
</Appenders>
<Loggers>
<Logger name="org.springframework" level="warn" additivity="false">
<appender-ref ref="${sys:logger.out.type}"/>
</Logger>
<Logger name="org.springframework.jdbc.core" level="off" additivity="false">
<appender-ref ref="${sys:logger.out.type}"/>
</Logger>
<Logger name="org.springframework.jdbc.core.StatementCreatorUtils" level="off" additivity="false">
<appender-ref ref="${sys:logger.out.type}"/>
</Logger>
<Logger name="com.myOrgName" level="trace" additivity="false">
<appender-ref ref="${sys:logger.out.type}"/>
</Logger>
<Root level="info" additivity="false">
<appender-ref ref="${sys:logger.out.type}"/>
</Root>
</Loggers>
</configuration>
In my previous StackOverflow question, a very kind user redirected me to the Log4j Property Substitution page to better understand the semantics of ${sys:logger.out.type}.
After reading through the documentation, I understand the sys Prefix is referring to a system property but don't quite understand what logger.out.type means?
Taking a stab here, is it referring to the output type of a Logger? Which could mean it's referring to the Console element named stdout?
Appreciate any insights & feedback from the community. Should any of my understanding be mistaken, please correct me, I am also learning :) Thank you!
A property like ${sys:logger.out.type} in a log4j 2.x configuration file resolves to the value of the Java system property named logger.out.type.
You set system properties by passing the command line argument -Dlogger.out.type=stdout (or more generally -D<property name>=<value>) to the JVM when you invoke the java command.
You can also provide defaults, so ${sys:logger.out.type:-FILE} would choose your FILE appender if no value was set for that system property.
I have a custom Scalar Valued function in SQL with the below details:
Name : fn_WorkDays
Returns : Int
Parameters : 1. StartDate (type - datetime) 2. EndDate (type - datetime)
I'm using Hibernate 5.4.6 to fetch some records from the database. I want to use the above function in the HQL (in the Where Clause), however currently facing issues.
Query<Device> query = session.createQuery("Select ud from Device ud where fn_WorkDays(ua.created, current_date())>2", Device.class);
After going through some posts, I understand that the custom function needs to be registered before using it, hence have created the below class:
public class MySQLServerDialect extends SQLServerDialect {
public MySQLServerDialect() {
super();
registerFunction("fn_WorkDays", new VarArgsSQLFunction(StandardBasicTypes.INTEGER, "fn_WorkDays(", ",", ")"));
}
}
Have updated the hibernate.cfg.xml with :
<property name="dialect">com.test.service.utils.MySQLServerDialect</property>
I dont seem to understand how the registerFunction should be done as im getting the below error:
ERROR SqlExceptionHelper:142 - 'fn_WorkDays' is not a recognized function name.
Log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<!-- Console Appender -->
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{yyyy-MMM-dd HH:mm:ss a} [%t] %-5level %logger{36}:%L - %msg%n" />
</Console>
</Appenders>
<Loggers>
<!-- Log everything in hibernate -->
<Logger name="org.hibernate" level="debug" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<!-- Log SQL statements -->
<Logger name="org.hibernate.SQL" level="debug" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<!-- Log JDBC bind parameters -->
<Logger name="org.hibernate.type.descriptor.sql" level="trace" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<!-- Log custom packages -->
<Logger name="com.test.service" level="debug" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<Root level="error">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
Any help to resolve the above will be greatly appreciated.
ERROR SqlExceptionHelper:142 - 'fn_WorkDays' is not a recognized function name.
The error already says what is wrong, the function does not exist in the database.
Try this:
registerFunction("fn_WorkDays", new VarArgsSQLFunction(StandardBasicTypes.INTEGER, "dbo.fn_WorkDays(", ",", ")"));
Functions that are not pre-defined in the JPQL/HQL are invoked like this:
`function('fn_WorkDays' , ua.created, current_date())`
This is later translated to fn_WorkDays(ua.created, current_date()) in the native query. You still need to have it registered by the dialect as you have.
I was trying to configure logback-spring.xml which will have loggers specific to environment.
This is what I was trying
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern name="basicPattern">My Pattern</pattern>
</encoder>
</appender>
<!--For Prod envs, only INFO levels are added -->
<springProfile name="prod | staging">
// Have only loggers with INFO e.g.
<logger name="org.springframework.web" level="INFO"/>
<logger name="org.hibernate.transaction" level="INFO"/>
</springProfile>
<!-- For all non-prod envs, it'll be DEBUG level -->
<logger name="org.springframework.web" level="DEBUG"/>
<logger name="org.hibernate.transaction" level="DEBUG"/>
<root level="DEBUG">
<appender-ref ref="CONSOLE"/>
</root>
Instead of having 2 or multiple <springProfiles> for each environment, how to configure for something specific & rest applicable to all environments?
I've tried this but didn't work. Is there some sequence which matters or do I have to duplicate all logs config in both <springProfiles>?
I am facing below error on upgradation of log4j 1.x to log4j2.
As per documentation on log4j2 site I have prepared below log4j2 file, but still I am getting this exception.
Log4j2 version: 2.6.1, Weblogic: 10.3.6, jdk 1.7.x
Please help with this exception:
2016-06-25 05:49:36,002 [ACTIVE] ExecuteThread: '70' for queue: 'weblogic.kernel.Default (self-tuning)' ERROR Unable to invoke factory method in class class org.apache.logging.log4j.core.config.LoggerConfig for element Logger. org.apache.logging.log4j.core.config.ConfigurationException: Arguments given for element Logger are invalid
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.generateParameters(PluginBuilder.java:270)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:131)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:918)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:858)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:850)
Log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG" >
<Properties>
<Property name="theHostName">${hostName}</Property>
</Properties>
<!-- bufferedIO=true by default, bufferSize=8192bytes
-->
<Appenders>
<RollingFile name="FILE" filename="${sys:weblogic.Name}.log" filepattern="${sys:weblogic.Name}.log.%i" append="true" >
<PatternLayout pattern="[%-5p][%d{yyyy-MM-dd HH:mm:ss,SSS}][${sys:weblogic.Name}:${hostName}][%t][%X{MessageInfo}][%c{1}:%M:%L][%msg]%n" />
<Policies>
<SizeBasedTriggeringPolicy size="50 MB" />
</Policies>
<DefaultRolloverStrategy max="100" fileIndex="min"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger level="DEBUG" includeLocation="true">
<AppenderRef ref="FILE"/>
</Logger>
<Root level="INFO" includeLocation="true">
<AppenderRef ref="FILE"/>
</Root>
<!-- Package specific log level defines -->
<logger level="WARN" name="org.springframework" />
<logger level="WARN" name="org.jboss" />
<logger level="OFF" name="org.hibernate" />
<logger level="WARN" name="com.company.project.eligibility" />
</Loggers>
</Configuration>
Only the root logger can have no name, so this Logger needs to have a name:
<Logger level="DEBUG" includeLocation="true">
<AppenderRef ref="FILE"/>
</Logger>
I guess package specific levels should be commented out or declared like this:
<Logger level="WARN" name="org.springframework" />
<Logger level="WARN" name="org.jboss" />
<Logger level="OFF" name="org.hibernate" />
<Logger level="WARN" name="com.company.project.eligibility" />
Is that possible to use MemoryMappedFile with Policies to break the files until fixed sizes, for example 250MB? My log4j2.xml is like this but I want to break the log files to 250MB and I need to use MemoryMappedFile for IO performance.
<Configuration monitorInterval="30">
<Appenders>
<MemoryMappedFile name="MemoryMap" fileName="output/jscsi-out.log">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{2} - %m%n(%L)" />
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="250 MB" />
</Policies>
</MemoryMappedFile>
</Appenders>
<Loggers>
<Logger name="br.com" level="debug" additivity="false">
<AppenderRef ref="MemoryMap" />
</Logger>
<Logger name="org.jscsi.target.TargetServer" level="debug"
additivity="false">
<AppenderRef ref="MemoryMap" />
</Logger>
<Logger name="org.jscsi.target.storage" level="info"
additivity="false">
<AppenderRef ref="MemoryMap" />
</Logger>
<Logger name="org.jscsi.service" level="debug" additivity="false">
<AppenderRef ref="MemoryMap" />
</Logger>
<Root level="error" includeLocation="true">
<AppenderRef ref="MemoryMap" />
</Root>
</Loggers>
</Configuration>
(Background: I am the author of the MemoryMappedFile appender as well as Async Loggers.)
In some sense the MemoryMappedFile appender is still a work in progress (as of Log4j 2.5). As you mention, there is no Rolling variant. Also, I haven't been able to do any significant performance testing. I made a start some time ago but other issues took priority.
Initial performance testing indicates that synchronous logging, even when using the MemoryMappedFile appender, is unlikely to be faster than asynchronous logging via Async Loggers.
Generally, if you need the rollover behaviour I would suggest you use Async Loggers in combination with the RollingRandomAccessFile appender for now.