I have Filebeat pulling logs from stdout. I want to ensure my logs are outputted as JSON, so they can be properly parsed.
Thus far, here's what I've found:
org.jboss.logmanager.formatters doesn't have a JSON formatter
There's an "extension" module that provides a JsonFormatter class.
I can use it in my logging.properties by doing something like this:
handler.CONSOLE=org.jboss.logmanager.handlers.ConsoleHandler
handler.CONSOLE.properties=autoFlush,target
handler.CONSOLE.autoFlush=true
handler.CONSOLE.formatter=JSON-FORMATTER
handler.CONSOLE.target=SYSTEM_OUT
formatter.JSON-FORMATTER=org.jboss.logmanager.ext.formatters.JSONFormatter
I need to know:
Am I missing anything with this configuration?
How can I customise the JSON output (i.e. add or remove fields)?
There is a json-formatter in WildFly 14. I would not suggest editing the logging.properties. The following CLI commands are an example of configuring a json-formatter.
/subsystem=logging/json-formatter=json:add(exception-output-type=formatted, pretty-print=false, meta-data={label=value})
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=named-formatter, value=json)
Note the meta-data attribute is just a key/value pair separated by commas.
How can I customise the JSON output (i.e. add or remove fields)?
You can really only add metadata or change field names. You can't remove fields though.
Related
I am developing a web application using the full stack framework focframework, and I want to know what are the properties that I can control in my config.properties file. Is there a doc for this?
I tried searching the doc but dodn't find anything
Obviously we can figure out some of then from the sample on GitHub by looking at the config.properties file:
jdbc.drivers=org.h2.Driver
jdbc.url=jdbc:h2:./myfocapplication_data_h2
jdbc.username=sa
jdbc.password=
gui.rtl=0
allowAddInsideComboBox=0
focWebServerClassName=com.focframework.sample.myfocapplication.MyFocAppWebServer
dataSourceClass=b01.focDataSourceDB.FocDataSource_DB
cloudStorageClass=com.focCloudStorage.FocCloudStorageS3
cloudStorageClass=com.foc.cloudStorage.FocCloudStorage_LocalDisc
devMode=1
unitDevMode=0
unitAllowed=1
log.dir=c:/01barmaja/log
log.ConsoleActive=1
log.fileActive=1
log.popupExceptionDialog=1
log.dbRequest=1
log.dbSelect=1
debug.showStatusColumn=0
log.debug=1
perf.active=0
Is there any hint on how to get all of them? And what if I want to add my own to be used in my code?
The ConfigInfo.java file is the one responsible of reading all the properties and storing them in variables. It is straight forward to understand and check the variables names and usage. Yet I agree that someone should work on the documentation and add these parameters.
To add your own without modifying the ConfigInfo.java you can simply use this method in the middle of your code.
String myProperty = ConfigInfo.getProperty("my.property.with.a.meanignful.name");
I am trying to eliminate a leading hyphen from our console and file logs in SpringBoot 1.3.5.RELEASE with default logback config.
Logging pattern looks like:
logging:
pattern:
console: '%d{yyyy-MM-dd HH:mm:ss.SSS} %clr([${spring.application.name}]){red} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %X{req.requestId} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}'
The file log pattern is similar, without the color coding. Both output every line after the first with a leading hyphen, which is making our syslog - logstash - grok filtering trickier. Example output:
2016-06-21 11:52:00.576 [my-app] INFO etc.. (application started)
-2016-06-21 11:52:00.583 [my-app] DEBUG etc..
-2016-06-21 11:52:00.583 [my-app] INFO etc..
I can't see anything in the docs mentioning this behaviour. Welcome any advice on how to eliminate it, if possible!
Update
Thanks to Edgar's answer below, it turns out this was caused by the following at the end of our logging pattern:
${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}
I replaced it with:
${LOG_EXCEPTION_CONVERSION_WORD:%rEx}
et voila, the hyphen at the start of the following line disappears. See http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-logging.html#boot-features-custom-log-configuration
The problem is in this part of logging.pattern.console:
${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}
:- is Logback's default value delimiter and is what you should use in logback.xml. However, you're configuring things in application.properties where it's Spring Framework's default value delimiter (:) which should be used.
As you've used :-, you're saying use the value of LOG_EXCEPTION_CONVERSION_WORD and, if it's not set, use -%rEx instead.
The correct syntax is:
${LOG_EXCEPTION_CONVERSION_WORD:%rEx}
It's hard to diagnose without your providing the full logging format. I'm seeing something similar in our code, and it seems to be related to including this in the format:
${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}
If you're using it, then the hyphen may be the one before the %w. I'm not sure what the intended purpose of this is. If I find it, I'll add it to my answer.
I want to use pgAdmin to configure to log all sql insert statements. But how?
I found the Backend Configuration Editor and enabled log_statement=all. But how can I control the file to that the log statements are written? (eg: c\logs.txt)
log_destination = 'stderr,csvlog'
log_statement = 'all'
#see C:\Program Files\PostgreSQL\9.3\data\pg_log
See the manual. You probably want to set log_directory and log_filename.
You cannot set individual log destinations for particular categories of log line - say, log statements to one file, other logs to another file. This would be a nice feature; if you have time, consider raising the idea on pgsql-hackers and after discussion submitting a suitable patch.
Most people use csvlog, or use text logs with a log_line_prefix set, then filter the logs with simple tools like grep to extract the desired information.
For statement data collection, also have a look at the pg_stat_statements extension.
We are developing accounting web application which needs specific logs to know what is happening, here
we are using java.util.logging api and some times we need to print some what lengthy messages but in our admin console the message is restricted to some length so we can not able to see entire message in console. We have decided that to remove package name which is prior to log message. Is this good idea?
If it is good please let me know how to do that?
Since Java 7 you can set the property java.util.logging.SimpleFormatter.format in your logging properties file.
So you could set up shorter messages by trying a different format.
For your specific case the format [%1$tc] %4$s: %2$s - %5$s %6$s%n will print the source field (package and class name), but the format [%1$tc] %4$s: %5$s %6$s%n will not.
This answer has an example of a logging.properties file. To use the new format use:
java.util.logging.SimpleFormatter.format = [%1$tc] %4$s: %5$s %6$s%n
I think what you need is a custom formatter which does this for you. Please refer http://docs.oracle.com/javase/7/docs/api/java/util/logging/Formatter.html
You can implement your own custom Log formatter, extending java.util.logging.Formatter
Here's an example
I have problems to read info from properties file "liferay-portal-6.1.0/tomcat-7.0.23/lib/myweb-application.properties" in portal_normal.vm
myweb-application.properties:
redirect.docs.url = http://stackoverflow.com/questions/ask
"liferay-portal-6.1.0/tomcat-7.0.23/webapps/web-theme/templates/portal_normal.vm":
#set ($docsURL = $propsUtil.get("redirect.docs.url")) #language("foot.docs")
Get value of portal-ext.properties
eg. test.name="sachin"
${propsUtil.get("test.name")}
PropsUtil (or $propsUtil) accesses portal.properties, typically configured through portal-ext.properties. So unless you add your myweb-application.properties as an "external properties" file, PropsUtil won't find it.
One way to do this is to add this line to your portal-ext.properties:
include-and-override=/path/to/myweb-application.properties
but make sure it's not using the same keys as portal.properties for different purposes.