Develop Reference

Any idea to prevent specific attribute to appear in the logs?

I'm looking for a way to prevent some sensitive data from being logged.
Ideally i would like to prevent / capture things like
String sensitive = "";
log.info ("This should be prevented or caught by something : {} ", sensitive);
this post is a bit of a longshot, I'm willing to investigate on any lead.
annotation, new types, Sonar Rules, logger hacking etc...
thx for your brainstorming :)
guillaume

Create custom type for it.
Make sure that toString doesn't return actual content.

I imagine there are multiple ways to do this, but one way is to use the Logback configuration file, to specify a message provider for the "arguments" and "message". In those providers, you define a "writeTo" method that looks for particular patterns in the output, and masks them.
This is the path to a solution, but I obviously don't provide many details here. I'm not aware of any "standard" solutions for this.
Another possibility would avail itself if your architecture has services running in transient containers, and the log output is sent to a centralized log aggregator, like Splunk. If you were ok with the initial logs written in the container having sensitive data, you could have the log aggregator look for patterns to mask for.

I would recommend two options, can you split your PII data into a separate log and then log that data securely?
If not, consider something like Cribl Logstream. Point your log shipper at it and let it strip away any PII you are concerned about. LogStream makes it very very easy to remove/mask/encrypt sensitive data. It has all sorts of other features as well.
At my last job we used LogStream as the router to make decisions about the data based on the content. PII data was detected and one copy was pushed to a secure PII certified logging platform and another copy was pushed to the operational logging platform but the PII data was masked so a wider audience could use the logging with no risk. It was a very useful workflow that solved a log of problems.

Retrieving previous n lines of log4j specific to one logger class

We implemented a small routine that in case of specific errors we get an email to let us know something happened. It's pretty easy to include some error information in the email but we would like to include what was the previous logged informations when it happened.
At first I tried to retrieve the last lines from the file where the log4j are saved. The problem is that many other threads are working at the same time and I never get relevant informations. Either the buffer doesn't have time to write the logs or something else is faster at writing something else.
Is it possible to add an appender to a specific log4j at runtime to be able to retrieve only those specific logs and only if needed ?
I would like to be able to still log with the usual commands log.error, log.warn... but I want to be able to retrieve the log content in a try/catch situation. something like a memory only buffer appender?
Would that work for all the children loggers ?
Class 1 - call Class 2
Class 2 - call Class 3
Class 3 try/catch an error
log.getLogs returns all logged informations from Class 1..3
Or am I dreaming here ?
We are using Jira and in logs in files, so I guess no query would work.

How do I log from a mapper? (hadoop with commoncrawl)

I'm using the commoncrawl example code from their "Mapreduce for the Masses" tutorial. I'm trying to make modifications to the mapper and I'd like to be able to log strings to some output. I'm considering setting up some noSQL db and just pushing my output to it, but it doesn't feel like a good solution. What's the standard way to do this kind of logging from java?

While there is no special solution for the logs aside of usual logger (at least one I am aware about) I can see about some solutions.
a) if logs are of debug purpose - indeed write usual debug logs. In case of the failed tasks you can find them via UI and analyze.
b) if this logs are some kind of output you want to get alongside some other output from you job - assign them some specail key and write to the context. Then in the reducer you will need some special logic to put them to the output.
c) You can create directory on HDFS and make mapper to write to there. It is not classic way for MR because it is side effect - in some cases it can be fine. Especially taking to account that after each mapper will create its own file - you can use command hadoop fs -getmerge ... to get all logs as one file.
c) If you want to be able to monitor the progress of your job, number of error etc - you can use counters.

slf4j How to specify max logged number for a line

I've got a huge number of log messages in project's log. I want to reduce that number by logging only first N of them for every line of code where log is used.
Are there any existent solutions for my task?
Thanks in advance.

This is not something i have tried myself, but if you use logback as the underlying logging mechanism of slf4j, I imagine you could write a custom filter that takes care of filtering the log output as you specify it.
http://logback.qos.ch/manual/filters.html

I decided to implement a LoggerWrapper by myself.
We can obtain the current line of a caller, count its occurrence and decide whether print a message or not. I think it's quite simple but a kind of routine.

How to validate Java logging properties files?

I have a basic facility for allowing users to remotely apply changes to the logging files in my application. Some logs are configured using java.util.logging properties files, and some are configured using log4j/log4cplus-style properties files. I'd like to do some basic validation of the properties that users try to apply. Namely, I want to assure the following:
Every logging.properties file must always contain at least a root logger/logging level
The logger/level must be set to a valid value. That is, they should not be able to set .level = GIBBERISH or anything like that.
I'll probably allow them to set MaxFileSize and MaxBackupIndex (log4j), and .limit and .count properties (java.util.logging), too.
What's the best way to accomplish this? I can obviously just loop over the keys and values in a Properties object and look for their values in a hard-coded Map or some other data structure that tells what valid properties are, but I'm trying to come up with a solution that's a little more elegant than that.

The problem with running any set of partial syntax checks against the properties files is that they'll always be inadequate by definition unless you capture every partial variation acceptable by the logging system, in which case you'll have recreated a portion of the logging system. No matter what properties you choose to validate theres bound to be additional ways to submit broken files.
Rather than testing for individual properties, why not create an additional (temporary, for the scope of the check only) logger object based on the input file and detect if it throws an error?

The "elegant" solution would be to write a rule-based engine for checking sets of name-value pairs. But IMO that is totally over the top for this use-case ... unless the checks are far more complex than I imagine.
I'd say that the simple (inelegant) solution is best in this case.