Develop Reference

Filter System.out.println messages in journalctl by Error, Warn, Info

How can I Filter Messages in Java, that is controlled by Systemd?
At the moment, I'm writing logs using System.out.println like this:
System.out.println("Hello this is a Error");
and the Journalctl -b -u test.service gives me this:
Nov 02 11:58:41 test java[10]: Hello this is a Error
I would like to do something like this
Consider a pseudocode:
System.out.println("Hello this is a Error", "Warn");
and obtain the following:
Nov 02 11:58:41 test java[10] [Error]: Hello this is a Error
But I don't want to use Log4j or any other Library. Maybe I just don't know what to search for, can't find a solution. Hope somebody has an idea.

System.err
I suspect you want something very simple, so you can try the error output stream accessible via System.err. It can be used pretty match like a standard output stream System.out with which you're already familiar.
You can create your own message patterns and apply them using printf() (refer to the official tutorial provided by Oracle for more information on string formatting).
System.err.println("message");
System.err.printf(LocalDate.now() + " : %s", "message");
The message would be printed on the Console, highlighted in red. That's it.
Java logging API
If you want something more elaborate, Java has a built-in logging API (no external dependencies required). It resides in the java.util.logging (that's why it's called JUL for short) and offers lots of functionality and many tools including Loggers, Handler, Filters, Formatters, etc. You can learn more about it here.
Here's a very dummy example:
Logger logger = Logger.getLogger("myLogger");
logger.log(Level.WARNING, "something is wrong");
That said, if you would decide to use JUL, you at least need to be aware what are the downsides of it and why other logging tool exist.
For instance, you can read this question:
Why not use java.util.logging?

Apache camel losing trace id and span id after camel split eip

I need to have trace id and span id available in all my logs. However I am observing that after the first splitter in my camel route, I can no longer see the trace id and span id in my logs.
[traceId: spanId:] INFO ---
Is there any way to enable back the tracing information?
From the Camel Documentation I have tried to start the tracing after the split by using
context.setTracing(true)
But looks like this is not working.
Am I missing anything, please help.

You probably have the traceId and spanId stored in the exchange message headers which are lost after the split.
A solution is to store them in the exchange properties(before the split) which are stored for the entire processing of the exchange(see Passing values between processors in apache camel).
If you are using the Java DSL you can use:
.setProperty("traceId ", constant("traceIdValue"))
.setProperty("spanId", constant("spanIdValue"))
You can use the Simple Expression Language(https://camel.apache.org/manual/latest/simple-language.html) to access the properties after the split using exchangeProperty.property_name.
Example:
.log(LoggingLevel.INFO, "[traceId:${exchangeProperty.traceId} spanId:${exchangeProperty.spanId}]")

When you use split, a new and old exchange will be created and to pass exchange properties downstream, you would need to use an aggregator to do so.
Example:
.split().tokenize(System.lineSeparator()).aggregationStrategy(new YourAggregationStrategyClass())

ESAPI not passing file validation

I am working with ESAPI to try and validate windows directory paths. For some reason, the part of my directory path named \14\ is getting converted into a CRLF. The error I am receiving is below, what am I not understanding correctly? I feel like my regex should be working.
WARN IntrusionDetector [SECURITY FAILURE Anonymous:null#unknown -> /project-test/IntrusionDetector] Invalid input: context=directoryPath, type(DirectoryName)=^[a-zA-Z0-9:/\\!##$%^&{}\[\]()_+\-=,.~'` ]{1,255}$, input=C:\UsersTESTUS~1AppDataLocalTempTestCase8002TempWorkSpace, orig=C:\Users\TESTUS~1\AppData\Local\Temp\14\TestCase8002TempWorkSpace
As you can see, I am using the regex:
^[a-zA-Z0-9:/\!##$%^&{}[]()_+-=,.~'` ]$
My input is:
C:\Users\TESTUS~1\AppData\Local\Temp\14\TestCase8002TempWorkSpace
Ouput, after ESAPI does canonicalization and validation:
C:\UsersTESTUS~1AppDataLocalTempTestCase8002TempWorkSpace
Here is the line of code that causes me to receive the error;
String validatedSourcePath = ESAPI.validator().getValidInput("directoryUnzip", directory, "DirectoryName", 255, false);
File validFile = new File(validatedSourcePath);

#C.Williams: I was about 30 minutes into writing up a detailed reply in an editor and accidentally excited my editor window. I'm too ticked off at my stupidity of not saving it to write it again, especially since I was only about 75% done.
However, if you want to email me I can arrange to talk to you via Google Hangouts or Signal to tell you want your problem is and how you can fix it. But it's long and complicated and partially related to a bug the ESAPI team just fixed but is not in any official release yet. But I am not going to take another 45 minutes or more trying to reply with any written text. My email address should be easy enough to find. Just google for my name and ESAPI. I am one of the project co-leaders on ESAPI.
-kevin wall

SpringBoot logging - extraneous hyphen at start of every log entry

I am trying to eliminate a leading hyphen from our console and file logs in SpringBoot 1.3.5.RELEASE with default logback config.
Logging pattern looks like:
logging:
pattern:
console: '%d{yyyy-MM-dd HH:mm:ss.SSS} %clr([${spring.application.name}]){red} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %X{req.requestId} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}'
The file log pattern is similar, without the color coding. Both output every line after the first with a leading hyphen, which is making our syslog - logstash - grok filtering trickier. Example output:
2016-06-21 11:52:00.576 [my-app] INFO etc.. (application started)
-2016-06-21 11:52:00.583 [my-app] DEBUG etc..
-2016-06-21 11:52:00.583 [my-app] INFO etc..
I can't see anything in the docs mentioning this behaviour. Welcome any advice on how to eliminate it, if possible!
Update
Thanks to Edgar's answer below, it turns out this was caused by the following at the end of our logging pattern:
${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}
I replaced it with:
${LOG_EXCEPTION_CONVERSION_WORD:%rEx}
et voila, the hyphen at the start of the following line disappears. See http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-logging.html#boot-features-custom-log-configuration

The problem is in this part of logging.pattern.console:
${LOG_EXCEPTION_CONVERSION_WORD:-%rEx}
:- is Logback's default value delimiter and is what you should use in logback.xml. However, you're configuring things in application.properties where it's Spring Framework's default value delimiter (:) which should be used.
As you've used :-, you're saying use the value of LOG_EXCEPTION_CONVERSION_WORD and, if it's not set, use -%rEx instead.
The correct syntax is:
${LOG_EXCEPTION_CONVERSION_WORD:%rEx}

It's hard to diagnose without your providing the full logging format. I'm seeing something similar in our code, and it seems to be related to including this in the format:
${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}
If you're using it, then the hyphen may be the one before the %w. I'm not sure what the intended purpose of this is. If I find it, I'll add it to my answer.

log forging fortify fix

I am using Fortify SCA to find the security issues in my application (as a university homework). I have encountered some 'Log Forging' issues which I am not able to get rid off.
Basically, I log some values that come as user input from a web interface:
logger.warn("current id not valid - " + bean.getRecordId()));
and Fortify reports this as a log forging issue, because the getRecordId() returns an user input.
I have followed this article, and I am replacing the 'new line' with space, but the issue is still reported
logger.warn("current id not valid - " + Util.replaceNewLine(bean.getRecordId()));
Can anyone suggest a way to fix this issue?

I know this was already answered, but I thought an example would be nice :)
<?xml version="1.0" encoding="UTF-8"?>
<RulePack xmlns="xmlns://www.fortifysoftware.com/schema/rules">
<RulePackID>D82118B1-BBAE-4047-9066-5FC821E16456</RulePackID>
<SKU>SKU-Validated-Log-Forging</SKU>
<Name><![CDATA[Validated-Log-Forging]]></Name>
<Version>1.0</Version>
<Description><![CDATA[Validated-Log-Forging]]></Description>
<Rules version="3.14">
<RuleDefinitions>
<DataflowCleanseRule formatVersion="3.14" language="java">
<RuleID>DDAB5D73-8CF6-45E0-888C-EEEFBEFF2CD5</RuleID>
<TaintFlags>+VALIDATED_LOG_FORGING</TaintFlags>
<FunctionIdentifier>
<NamespaceName>
<Pattern/>
</NamespaceName>
<ClassName>
<Pattern>Util</Pattern>
</ClassName>
<FunctionName>
<Pattern>replaceNewLine</Pattern>
</FunctionName>
<ApplyTo implements="true" overrides="true" extends="true"/>
</FunctionIdentifier>
<OutArguments>return</OutArguments>
</DataflowCleanseRule>
</RuleDefinitions>
</Rules>
</RulePack>

Alina, I'm actually the author of the article you used to solve your log injection issue. Hope it was helpful.
Vitaly is correct with regards to Fortify. You'll need to build what Fortify calls a "custom rule".
It will likely be a dataflow cleanse rule. A basic example can be found here: http://www.cigital.com/newsletter/2009-11-tips.php. If you own Fortify, there should be a custom rule writing guide in your product documentation.
I don't know what the taint flag you'll use is, but it would look something like "-LOG_FORGING". You would essentially write a rule to remove the log forging "taint" whenever data is passed through your utility method. Fortify will them assume that any data passed through there is now safe to be written to a log, and will not cause log forging.

You need to mark your replaceNewLine as sanitiser in Fortify (if I remember correctly) and it will stop reporting the issue.

You can actually create a new rule from a particular method.
Navigate to the function on the right side of audit workbench after you've done a scan.
Find your sanitizing method and right click on it.
You can generate a rule from it. What you want is a general DataflowCleanseRule.
I just did this based on the xml someone posted above. You can save the rule as a .xml file.
When updating your scan you can pass the -rule argument and point at the .xml file.