I'll put the question upfront:
Is there a logger available in Java that does encryption(preferably 128-bit AES or better)?
I've done a lot of searching for this over the last couple of days. There's a few common themes to what I've found:
Dissecting information between log4j and log4j2 is giving me headaches(but mostly unrelated to the task at hand)
Most threads are dated, including the ones here on SO. This one is probably the best I've found on SO, and one of the newer answers links to a roll-your-own version.
The most common answer is "roll-your-own", but these answers are also a few years old at this point.
A lot of people question why I or anyone would do this in Java anyway, since it's simple enough to analyze Java code even without the source.
For the last point, it's pretty much a moot point for my project. We also use a code obfuscator and could employ other obfuscation techniques. The point of using encryption is simply to raise the bar of figuring out our logs above "trivially easy", even if it's only raised to "mildly time-consuming". A slightly relevant aside - the kind of logging we're going to encrypt is intended merely for alpha/beta, and will likely only include debug, warn, and error levels of logging(so the number of messages to encrypt should be fairly low).
The best I've found for Log4j2 is in their documentation:
KeyProviders
Some components within Log4j may provide the ability to perform data encryption. These components require a secret key to perform the encryption. Applications may provide the key by creating a class that implements the SecretKeyProvider interface.
But I haven't really found anything other than wispy statements along the lines of 'plug-ins are able of doing encryption'. I haven't found a plug-in that actually has that capability.
I have also just started trying to find other loggers for Java to see if they have one implemented, but nothing is really jumping out for searches like 'java logging encryption'.
Basically log encryption is not best practise there are limited situations where you can need this functionality. As mainly people which have access to logs have also access to JVM, and in JVM all the logs are at least generated as Strings so even if you encrypt them in the log file or console the real values will be available in JVM String Pool, so if anyone will every need to hack your logs it will be as easy as have a look in string pool.
But anyway if you need a way to encrypt the logs, and as there is no generic way for this, the best way in my opinion is to go with Aspect J. This will have minimum impact on you sources, you will write code as you have done before, but the logs will be encrypted. Following is a simple application code which will encrypt all the logs from all the compiled sources using Aspctj, and Slf4j as logging facade and Log4j2 as logging implementation.
The simple class which logs the "Hello World"
public class Main {
private static final transient Logger LOG = LoggerFactory
.getLogger(Main.class);
public static void main(String[] args) {
LOG.info("Hello World");
LOG.info("Hello {0}", "World 2");
}
}
Aspect which encrypts (in this case just edits the text)
#Aspect
public class LogEncryptAspect {
#Around("call(* org.slf4j.Logger.info(..))")
public Object encryptLog (ProceedingJoinPoint thisJoinPoint) throws Throwable{
Object[] arguments = thisJoinPoint.getArgs();
if(arguments[0] instanceof String){
String encryptedLog = encryptLogMessage ((String) arguments[0], arguments.length > 1 ? Arrays.copyOfRange(arguments, 1, arguments.length) : null);
arguments[0] = encryptedLog;
}
return thisJoinPoint.proceed(arguments);
}
// TODO change this to apply some kind of encryption
public final String encryptLogMessage (String message, Object... args){
if(args != null){
return MessageFormat.format(message, args) + " encrypted";
}
return message + " encrypted";
}
}
The output is :
[main] INFO xxx.Main - Hello World encrypted
[main] INFO xxx.Main - Hello World 2 encrypted
Related
Any Android App produces Logs in the LogCat, even those not generated by developer's source code via Log.d, Log.i, Log.w and Log.e etc. etc. etc.. Perhaps Google Developers has some "automagic" thing for this, I don't know about that...
The point is I remember, years ago, I could somehow extend the class Application, override one or several of it's methods, and then:
Add my own code to process any single Log object generated by my
App in the LogCat
Do whatever I wanted with them (getting the label and the description strings, and then send them via mail, Slack etc., basically)
And then, calling super on that method and let the system do with that Log whatever Application by default does with it...
or something like that... if I recall correctly, I could do this with any log in my app's namespace. Or maybe it was just the crash handler? I can't remember...
It's been so long since I accomplished that (several years already!), so I don't remember how could I do that anymore... I search the internet like crazy trying to recall, but I am struggling to find it again... :-S
// ...public?? oO
[¿¿??] class MyApp extends Application [...] {
// [...]
#Override
public void whateverMethodItWasIDontRemember(params) {
// My coding stuff for the error reports
/* magic :D */
sendTheLogsMyWay();
// I bet this is important
super.whateverMethodItWasIDontRemember(params);
}
// [...]
}
I am about to launch the first Beta version of a new app, so I want beta testers to have a reliable way to send me LogCat's feed if anything has to be reported due to crashes, unexpected behaviour etc.
I mean, it would be ridiculous having to fill with CustomLogs every inch of source code for the beta version, when, in most cases, default logs are more than enough to see why it crashed (errors), or what optimization problems (usually warnings) might the Beta Tester have... not to mention that, if I forget to monitor something this way, the ridiculously big effort to log every single line of my code would be useless... oO
// -__- Mmm... perhaps extending Log itself
// would be more elegant...
import android.util.Log
public final class CustomLog {
public static void d(String label, String msg) {
// AKA My code to handle it
packItForNextErrorReport(label, msg);
Log.d(label, msg);
}
/*
* ... and so on with Log.i, w and e.
* ...I think you get the idea
*/
}
I am using log4j v1.2.14 for logging in my project and I am also using Java 7 String.format() to put variables in my output. Currently I am writing
LOGGER.info(String.format("Your var is [%s] and you are [%s]", myVar, myVar1));
Is this really the best way to output strings? I feel that log4j should have this implemented implicitly as below:
LOGGER.info("Your var is [%s] and you are [%s]", myVar, myVar1);
Have I missed something? Further, are there any Java logging frameworks that support this?
slf4j's api provides "parameterized logging", which allows you to do exactly that, although with a slightly different syntax. The example there is:
logger.debug("Value {} was inserted between {} and {}.", newVal, below, above);
For an implementation, you can use Logback which implements slf4j natively, or the slf4j bindings to connect with log4j or other loggers. The User Manual explains that, along with a short example.
Using String.format, +, or a string formatter other than the one provided by your logging system (log4j for example) is considered as a bad practice.
Usually, in the code there are lots of low level logs (debug, info) you don't want to see in production. If you use for example String.format to format the string to log, then you will allocate on the heap and format a new String, which can be very long and consume resources, even if at the end nothing will be logged (for example if the log4j min level is set to warning or error).
By using the logger formatter system (like the one from log4j), you allow your logger to avoid the generation of the formatted string if it doesn't need to be logged.
This may make a great difference in some cases.
Log4j supports internal formatting. I haven't found it documented anywhere, but I saw an example of it here:
https://logging.apache.org/log4j/2.x/manual/markers.html
I tried it out and it works! I'm on log4j 2.11.2.
int i = 42;
String str1 = "the answer";
String str2 = "life, the universe, and everything";
console.info("{} is {} to {}", i, str1, str2);
Looking at the javadoc for Logger, I'd say it was introduced in Lo4j 2, and supports up to 10 parameters.
https://logging.apache.org/log4j/2.x/log4j-api/apidocs/org/apache/logging/log4j/Logger.html
Btw, In this scenario there is not much difference between using + to add your variables to the string and String.format - unless you really want to reuse the "Your var is..." in all your logs.
slf4j lets you log as
log.info("Your var is {} and you are {}", myVar, myVar1);
Note the use of {} instead of print formatters. Also this requires Java >= 1.5
I upvoted the "use logger.debug(format, varargs)" approach first, because it doesn't allocate extra String when it's not needed.
But then it occurred to me that logger.debug(String format, Object... varargs) still allocates an Array for the varargs.
Tried this on https://godbolt.org/noscript/java
class VarargTest {
static boolean debugging = false;
static void debug(String format, Object... args) {
if (debugging) {
System.out.println(String.format(format, args));
}
}
static void logVarargs(int n) {
debug("debug message {}", n);
}
static void logIf(int n) {
if (VarargTest.debugging) {
debug("debug message 2 " + n);
}
}
}
and indeed, the resulting debug() call allocates an Array.
So the fastest code should be
if (logger.isDebugEnabled()) {
logger.debug("format {}", arg);
}
If it's not super performance critical, a more readable and reasonably fast code would be simply
logger.debug("format {}", arg);
I need to write my own logging handler on GAE/J. I have Android code that I'm trying to adapt such that it can be shared between GAE/J and Android. The GAE code I'm trying to write would allow the log statements in my existing code to work on GAE.
The docs say that I can just print to system.out and system.err, and it works, but badly. My logging shows up in the log viewer with too much extraneous text:
2013-03-08 19:37:11.355 [s~satethbreft22/1.365820955097965155].: [my_log_msg]
So, I started looking at the GAE log API. This looked hopeful initially: I can construct an AppLogLine and set the log records for a RequestLogs object.
However, there is no way to get the RequestLogs instance for the current request - the docs say so explicitly here:
Note: Currently, App Engine doesn't support the use of the request ID to directly look up the related logs.
I guess I could invent a new requestID and add log lines to that, but it is starting to look like this is just not meant to be?
Has anyone used this API to create their own log records, or otherwise managed to do their own logging to the log console.
Also, where can I find the source for GAE's java.util.logging? Is this public? I would like to see how that works if I can.
If what I'm trying to do is impossible then I will need to consider other options, e.g. writing my log output to a FusionTable.
I ended up just layering my logging code on top of GAE's java.util.logging. This feels non-optimal since it increases the complexity and overhead of my logging, but I guess this is what any 3rd logging framework for GAE must do (unless it is OK with the extra cruft that gets added when you just print to stdout).
Here is the crux of my code:
public int println(int priority, String msg) {
Throwable t = new Throwable();
StackTraceElement[] stackTrace = t.getStackTrace();
// Optional: translate from Android log levels to GAE log levels.
final Level[] levels = { Level.FINEST, Level.FINER, Level.FINE, Level.CONFIG,Level.INFO, Level.WARNING, Level.SEVERE, Level.SEVERE };
Level level = levels[priority];
LogRecord lr = new LogRecord(level, msg);
if (stackTrace.length > 2) { // should always be true
lr.setSourceClassName(stackTrace[2].getClassName());
lr.setSourceMethodName(stackTrace[2].getMethodName());
}
log.log(lr);
return 0;
}
Note that I use a stack depth of 2, but that # will depend on the 'depth' of your logging code.
I hope that Google will eventually support getting the current com.google.appengine.api.log.RequestLogs instance and inserting our own AppLogLine instances into it. (The API's are actually there to do that, but they explicitly don't support it, as above.)
I need a way to essentially secure my jar library to allow registered apps to use it in their projects and deny usage to apps that weren't approved by me.
It is fine if I hard code things in the lib for each distribution. I currently have this jar obfuscated.
What are good approaches to restrict the usage of a jar?
One idea was to lock the lib to a specific package so if the developer tries to use it in another project they can't. But I'm not sure if they can easily provide a custom fake Context to make it work...
To me the best approach if you would like your library to stay standalone (without involving the network for checking or downloading pieces of the library, I mean) would be to make mandatory the use of an initializer class that would receive a token from the client application.
This would be crackable as the token validity test would be performed by your lib: one may modify the lib in a way is would just skip that test, but this would be made harder by the obfuscation. But this is probably sufficient, unless using your lib without having registered it is a really critical issue.
So you would have something like:
boolean Initializer.initLib(String passcode)
That would prevent the lib to work unless passcode is correct.
You can make the obfuscation more efficient by avoiding checking that way:
public void initLib(String passcode) {
if (passcode == A_GIVEN_PUBLIC_STATIC_THAT_STORESTHE_CODE) {
// do the proper initializations
}
else {
throw new RuntimeException("Bad passcode, sorry!");
}
}
But doing that way instead:
public void initLib(String passcode) {
final char[] PASS_ENCRYPTED = "f5uhjgf56ik8kv214d5".toCharArray();
final char[] PASS_MINUSMASK = "bc".toCharArray();
final int PASS_SHIFT = 11;
final int PASS_MASK_MINUS = 2;
for (int ctr = 0; ctr < PASS_MINUSMASK.length; ++ctr) {
final char next = PASS_ENCRYPTED[PASS_SHIFT + ctr - PASS_MASK_MINUS];
if (passcode.charAt(ctr) != next - (PASS_MINUSMASK[ctr] - 'a')) {
// make the lib unusable by some inits. But it should look as a proper initialization
return;
}
}
// make the lib usable by some inits.
}
This looks stupid, but if you have a look at the obfuscated code, you will see a big difference. This code is just an example (it accepts "hi" as a valid passcode), any algorithm would be fine as long as its obfuscated version is not too straightforward to reverse.
Now the question is: what passcode?
As the library's protection concerns the developpers of the client apps that will use it, and not the final users of these apps, you cannot rely on any piece of data specific to the devices on which the applications will run. So no IMEI or anything like that.
If these developpers are trustworthy that's fine. A fixed passcode is sufficient.
But if they are subject to give this passcode to other people to allow them using your library, this is more difficult. In this case I don't think you can solve it without a real "industrial" process such as registering the client apps and their code checksums, for example. Such a process needs a specific design and cannot be solved "just by the code", but as it also has a cost (time, resources, involvment of the client...) you can only consider this if the use of library is very critical.
Can't you make your jar call your server with a specific code and the application name, to check if they are registered ?
When you build an Android app with a jar, that jar is compiled into the app and becomes a part of it. You can't just copy the jar out of the package and use it elsewhere. Unless I'm not understanding the question, this shouldn't be an issue you need to worry about.
Our web app needs to be made PCI compliant, i.e. it must not store any credit card numbers. The app is a frontend to a mainframe system which handles the CC numbers internally and - as we have just found out - occasionally still spits out a full CC number on one of its response screens. By default, the whole content of these responses are logged at debug level, and also the content parsed from these can be logged in lots of different places. So I can't hunt down the source of such data leaks. I must make sure that CC numbers are masked in our log files.
The regex part is not an issue, I will reuse the regex we already use in several other places. However I just can't find any good source on how to alter a part of a log message with Log4J. Filters seem to be much more limited, only able to decide whether to log a particular event or not, but can't alter the content of the message. I also found the ESAPI security wrapper API for Log4J which at first sight promises to do what I want. However, apparently I would need to replace all the loggers in the code with the ESAPI logger class - a pain in the butt. I would prefer a more transparent solution.
Any idea how to mask out credit card numbers from Log4J output?
Update: Based on #pgras's original idea, here is a working solution:
public class CardNumberFilteringLayout extends PatternLayout {
private static final String MASK = "$1++++++++++++";
private static final Pattern PATTERN = Pattern.compile("([0-9]{4})([0-9]{9,15})");
#Override
public String format(LoggingEvent event) {
if (event.getMessage() instanceof String) {
String message = event.getRenderedMessage();
Matcher matcher = PATTERN.matcher(message);
if (matcher.find()) {
String maskedMessage = matcher.replaceAll(MASK);
#SuppressWarnings({ "ThrowableResultOfMethodCallIgnored" })
Throwable throwable = event.getThrowableInformation() != null ?
event.getThrowableInformation().getThrowable() : null;
LoggingEvent maskedEvent = new LoggingEvent(event.fqnOfCategoryClass,
Logger.getLogger(event.getLoggerName()), event.timeStamp,
event.getLevel(), maskedMessage, throwable);
return super.format(maskedEvent);
}
}
return super.format(event);
}
}
Notes:
I mask with + rather than *, because I want to tell apart cases when the CID was masked by this logger, from cases when it was done by the backend server, or whoever else
I use a simplistic regex because I am not worried about false positives
The code is unit tested so I am fairly convinced it works properly. Of course, if you spot any possibility to improve it, please let me know :-)
You could write your own layout and configure it for all appenders...
Layout has a format method which makes a String from a loggingEvent that contains the logging message...
A better implementation of credit card number masking is at http://adamcaudill.com/2011/10/20/masking-credit-cards-for-pci/ .
You want to log the issuer and the checksum, but not the PAN (Primary Account Number).