Tomcat Guice/JDBC Memory Leak - java

I'm experiencing a memory leak due to orphaned threads in Tomcat. Particularly, it seems that Guice and the JDBC driver are not closing threads.
Aug 8, 2012 4:09:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: A web application appears to have started a thread named [com.google.inject.internal.util.$Finalizer] but has failed to stop it. This is very likely to create a memory leak.
Aug 8, 2012 4:09:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: A web application appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
I know this is similar to other questions (such as this one), but in my case, the answer of "don't worry about it" won't be sufficient, as it is causing problems for me. I have CI server which regularly updates this application, and after 6-10 reloads, the CI server will hang because Tomcat is out of memory.
I need to be able to clear up these orphaned threads so I can run my CI server more reliably. Any help would be appreciated!

I just dealt with this problem myself. Contrary to some other answers, I do not recommend issuing the t.stop() command. This method has been deprecated, and for good reason. Reference Oracle's reasons for doing this.
However there is a solution for removing this error without needing to resort to t.stop()...
You can use most of the code #Oso provided, just replace the following section
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")) {
synchronized(t) {
t.stop(); //don't complain, it works
}
}
}
Replace it using the following method provided by the MySQL driver:
try {
AbandonedConnectionCleanupThread.shutdown();
} catch (InterruptedException e) {
logger.warn("SEVERE problem cleaning up: " + e.getMessage());
e.printStackTrace();
}
This should properly shutdown the thread, and the error should go away.

I've had the same issue, and as Jeff says, the "don't worry about it approach" was not the way to go.
I did a ServletContextListener that stops the hung thread when the context is being closed, and then registered such ContextListener on the web.xml file.
I already know that stopping a thread is not an elegant way to deal with them, but otherwise the server keeps on crashing after two or three deploys (it is not always possible to restart the app server).
The class I created is:
public class ContextFinalizer implements ServletContextListener {
private static final Logger LOGGER = LoggerFactory.getLogger(ContextFinalizer.class);
#Override
public void contextInitialized(ServletContextEvent sce) {
}
#Override
public void contextDestroyed(ServletContextEvent sce) {
Enumeration<Driver> drivers = DriverManager.getDrivers();
Driver d = null;
while(drivers.hasMoreElements()) {
try {
d = drivers.nextElement();
DriverManager.deregisterDriver(d);
LOGGER.warn(String.format("Driver %s deregistered", d));
} catch (SQLException ex) {
LOGGER.warn(String.format("Error deregistering driver %s", d), ex);
}
}
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")) {
synchronized(t) {
t.stop(); //don't complain, it works
}
}
}
}
}
After creating the class, then register it on the web.xml file:
<web-app...
<listener>
<listener-class>path.to.ContextFinalizer</listener-class>
</listener>
</web-app>

The least invasive workaround is to force initialisation of the MySQL JDBC driver from code outside of the webapp's classloader.
In tomcat/conf/server.xml, modify (inside the Server element):
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
to
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
classesToInitialize="com.mysql.jdbc.NonRegisteringDriver" />
With mysql-connector-java-8.0.x use com.mysql.cj.jdbc.NonRegisteringDriver instead
This assumes you put the MySQL JDBC driver into tomcat's lib directory and not inside your webapp.war's WEB-INF/lib directory, as the whole point is to load the driver before and independently of your webapp.
References:
http://bugs.mysql.com/bug.php?id=68556#c400606
http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#JRE_Memory_Leak_Prevention_Listener_-_org.apache.catalina.core.JreMemoryLeakPreventionListener
http://markmail.org/message/dmvlkps7lbgpngil
com.mysql.jdbc.NonRegisteringDriver source v5.1
com.mysql.cj.jdbc.NonRegisteringDriver source v8.0
Changes in connector/J v8.0

Effective from MySQL connector 5.1.23 onwards, a method is provided to shut the abandoned connection cleanup thread down, AbandonedConnectionCleanupThread.shutdown.
However, we don't want direct dependencies in our code on the otherwise opaque JDBC driver code, so my solution is to use reflection to find the class and method and invoke it if found. The following complete code snippet is all that's needed, executed in the context of the class loader that loaded the JDBC driver:
try {
Class<?> cls=Class.forName("com.mysql.jdbc.AbandonedConnectionCleanupThread");
Method mth=(cls==null ? null : cls.getMethod("shutdown"));
if(mth!=null) { mth.invoke(null); }
}
catch (Throwable thr) {
thr.printStackTrace();
}
This cleanly ends the thread if the JDBC driver is a sufficiently recent version of the MySQL connector and otherwise does nothing.
Note it has to be executed in the context of the class loader because the thread is a static reference; if the driver class is not being or has not already been unloaded when this code is run then the thread will not be running for subsequent JDBC interactions.

I took the best parts of the answers above and combined them into an easily extensible class. This combines Oso's original suggestion with Bill's driver improvement and Software Monkey's reflection improvement. (I liked the simplicity of Stephan L's answer too, but sometimes modifying the Tomcat environment itself is not a good option, especially if you have to deal with autoscaling or migration to another web container.)
Instead of directly referring to the class name, thread name, and stop method, I also encapsulated these into an private inner ThreadInfo class. Using a list of these ThreadInfo objects, you can include additional troublesome threads to be shutdown with the same code. This is a bit more complex of a solution than most people likely need, but should work more generally when you need that.
import java.lang.reflect.Method;
import java.sql.Driver;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Set;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Context finalization to close threads (MySQL memory leak prevention).
* This solution combines the best techniques described in the linked Stack
* Overflow answer.
* #see Tomcat Guice/JDBC Memory Leak
*/
public class ContextFinalizer
implements ServletContextListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(ContextFinalizer.class);
/**
* Information for cleaning up a thread.
*/
private class ThreadInfo {
/**
* Name of the thread's initiating class.
*/
private final String name;
/**
* Cue identifying the thread.
*/
private final String cue;
/**
* Name of the method to stop the thread.
*/
private final String stop;
/**
* Basic constructor.
* #param n Name of the thread's initiating class.
* #param c Cue identifying the thread.
* #param s Name of the method to stop the thread.
*/
ThreadInfo(final String n, final String c, final String s) {
this.name = n;
this.cue = c;
this.stop = s;
}
/**
* #return the name
*/
public String getName() {
return this.name;
}
/**
* #return the cue
*/
public String getCue() {
return this.cue;
}
/**
* #return the stop
*/
public String getStop() {
return this.stop;
}
}
/**
* List of information on threads required to stop. This list may be
* expanded as necessary.
*/
private List<ThreadInfo> threads = Arrays.asList(
// Special cleanup for MySQL JDBC Connector.
new ThreadInfo(
"com.mysql.jdbc.AbandonedConnectionCleanupThread", //$NON-NLS-1$
"Abandoned connection cleanup thread", //$NON-NLS-1$
"shutdown" //$NON-NLS-1$
)
);
#Override
public void contextInitialized(final ServletContextEvent sce) {
// No-op.
}
#Override
public final void contextDestroyed(final ServletContextEvent sce) {
// Deregister all drivers.
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver d = drivers.nextElement();
try {
DriverManager.deregisterDriver(d);
LOGGER.info(
String.format(
"Driver %s deregistered", //$NON-NLS-1$
d
)
);
} catch (SQLException e) {
LOGGER.warn(
String.format(
"Failed to deregister driver %s", //$NON-NLS-1$
d
),
e
);
}
}
// Handle remaining threads.
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for (Thread t:threadArray) {
for (ThreadInfo i:this.threads) {
if (t.getName().contains(i.getCue())) {
synchronized (t) {
try {
Class<?> cls = Class.forName(i.getName());
if (cls != null) {
Method mth = cls.getMethod(i.getStop());
if (mth != null) {
mth.invoke(null);
LOGGER.info(
String.format(
"Connection cleanup thread %s shutdown successfully.", //$NON-NLS-1$
i.getName()
)
);
}
}
} catch (Throwable thr) {
LOGGER.warn(
String.format(
"Failed to shutdown connection cleanup thread %s: ", //$NON-NLS-1$
i.getName(),
thr.getMessage()
)
);
thr.printStackTrace();
}
}
}
}
}
}
}

I went a step further from Oso,improved the code above in two points:
Added the Finalizer thread to the need-to-kill check:
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")
|| t.getName().matches("com\\.google.*Finalizer")
) {
synchronized(t) {
logger.warn("Forcibly stopping thread to avoid memory leak: " + t.getName());
t.stop(); //don't complain, it works
}
}
}
Sleep for a little while to give threads time to stop. Without that, tomcat kept complaining.
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
logger.debug(e.getMessage(), e);
}

Bill's solution looks good, however I found another solution directly in MySQL bug reports:
[5 Jun 2013 17:12] Christopher Schultz
Here is a much better workaround until something else changes.
Enable Tomcat's JreMemoryLeakPreventionListener (enabled by default on Tomcat 7), and add this attribute to the element:
classesToInitialize="com.mysql.jdbc.NonRegisteringDriver"
If "classesToInitialize" is already set on your , just add NonRegisteringDriver to the existing value separated by a comma.
and the answer:
[8 Jun 2013 21:33] Marko Asplund
I did some testing with the JreMemoryLeakPreventionListener / classesToInitialize workaround (Tomcat 7.0.39 + MySQL Connector/J 5.1.25).
Before applying the workaround thread dumps listed multiple AbandonedConnectionCleanupThread instances after redeploying the webapp several times. After applying the workaround there's only one AbandonedConnectionCleanupThread instance.
I had to modify my app, though, and move MySQL driver from the webapp to Tomcat lib.
Otherwise, the classloader is unable to load com.mysql.jdbc.NonRegisteringDriver at Tomcat startup.
I hope it helps for all who still fighting with this issue...

It seems this was fixed in 5.1.41. You could upgrade Connector/J to 5.1.41 or newer.
https://dev.mysql.com/doc/relnotes/connector-j/5.1/en/news-5-1-41.html
The implementation of AbandonedConnectionCleanupThread has now been improved, so that there are now four ways for developers to deal with the situation:
When the default Tomcat configuration is used and the Connector/J jar is put into a local library directory, the new built-in application detector in Connector/J now detects the stopping of the web application within 5 seconds and kills AbandonedConnectionCleanupThread. Any unnecessary warnings about the thread being unstoppable are also avoided. If the Connector/J jar is put into a global library directory, the thread is left running until the JVM is unloaded.
When Tomcat's context is configured with the attribute clearReferencesStopThreads="true", Tomcat is going to stop all spawned threads when the application stops unless Connector/J is being shared with other web applications, in which case Connector/J is now protected against an inappropriate stop by Tomcat; the warning about the non-stoppable thread is still issued into Tomcat's error log.
When a ServletContextListener is implemented within each web application that calls AbandonedConnectionCleanupThread.checkedShutdown() on context destruction, Connector/J now, again, skips this operation if the driver is potentially shared with other applications. No warning about the thread being unstoppable is issued to Tomcat's error log in this case.
When AbandonedConnectionCleanupThread.uncheckedShutdown() is called, the AbandonedConnectionCleanupThread is closed even if Connector/J is shared with other applications. However, it may not be possible to restart the thread afterwards.
If you look at source code, they called setDeamon(true) on thread, so it won't block shutdown.
Thread t = new Thread(r, "Abandoned connection cleanup thread");
t.setDaemon(true);

See To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Bill's answer deregisters all Driver instances as well as instances that may belong to other web applications. I have extended Bill's answer with a check that the Driver instance belongs to the right ClassLoader.
Here is the resulting code (in a separate method, because my contextDestroyed has other things to do):
// See https://stackoverflow.com/questions/25699985/the-web-application-appears-to-have-started-a-thread-named-abandoned-connect
// and
// https://stackoverflow.com/questions/3320400/to-prevent-a-memory-leak-the-jdbc-driver-has-been-forcibly-unregistered/23912257#23912257
private void avoidGarbageCollectionWarning()
{
ClassLoader cl = Thread.currentThread().getContextClassLoader();
Enumeration<Driver> drivers = DriverManager.getDrivers();
Driver d = null;
while (drivers.hasMoreElements()) {
try {
d = drivers.nextElement();
if(d.getClass().getClassLoader() == cl) {
DriverManager.deregisterDriver(d);
logger.info(String.format("Driver %s deregistered", d));
}
else {
logger.info(String.format("Driver %s not deregistered because it might be in use elsewhere", d.toString()));
}
}
catch (SQLException ex) {
logger.warning(String.format("Error deregistering driver %s, exception: %s", d.toString(), ex.toString()));
}
}
try {
AbandonedConnectionCleanupThread.shutdown();
}
catch (InterruptedException e) {
logger.warning("SEVERE problem cleaning up: " + e.getMessage());
e.printStackTrace();
}
}
I wonder whether the call AbandonedConnectionCleanupThread.shutdown() is safe. Can it interfere with other web applications? I hope not, because the AbandonedConnectionCleanupThread.run() method is not static but the AbandonedConnectionCleanupThread.shutdown() method is.

Related

What is the CDI equivalent of EJB's SessionSynchronization#afterCompletion method?

I've read CDI 2.0 specification (JSR 365) and found out the existence of the #Observes(during=AFTER_SUCCESS) annotation, but it actually requires a custom event to be defined in order to work.
This is what i've got:
//simple """transactional""" file system manager using command pattern
#Transactional(value = Transactional.TxType.REQUIRED)
#TransactionScoped
#Stateful
public class TransactionalFileSystemManager implements SessionSynchronization {
private final Deque<Command> commands = new ArrayDeque<>();
public void createFile(InputStream content, Path path, String name) throws IOException {
CreateFile command = CreateFile.execute(content, path, name);
commands.addLast(command);
}
public void deleteFile(Path path) throws IOException {
DeleteFile command = DeleteFile.execute(path);
commands.addLast(command);
}
private void commit() throws IOException{
for(Command c : commands){
c.confirm();
}
}
private void rollback() throws IOException{
Iterator<Command> it = commands.descendingIterator();
while (it.hasNext()) {
Command c = it.next();
c.undo();
}
}
#Override
public void afterBegin() throws EJBException{
}
#Override
public void beforeCompletion() throws EJBException{
}
#Override
public void afterCompletion(boolean commitSucceeded) throws EJBException{
if(commitSucceeded){
try {
commit();
} catch (IOException e) {
throw new EJBException(e);
}
}
else {
try {
rollback();
} catch (IOException e) {
throw new EJBException(e);
}
}
}
}
However, I want to adopt a CDI-only solution so I need to remove anything EJB related (including the SessionSynchronization interface). How can i achieve the same result using CDI?
First the facts: the authoritative source for this topic is the Java Transaction API (JTA) specification. Search for it online, I got this.
Then the bad news: In order to truly participate in a JTA transaction, you either have to implement a connector according to the Java Connector Architecture (JCA) specification or a XAResource according to JTA. Never done any of them, I am afraid both are going to be hard. Nevertheless, if you search, you may find an existing implementation of a File System Connector.
Your code above will never accomplish true 2-phase commit because, if your code fails, the transaction is already committed, so the application state is inconsistent. Or, there is a small time window when the real transaction is committed but the file system change have not beed executed, again the state is inconsistent.
Some workarounds I can think of, none of which solves the consistency problem:
Persist the File System commands in a database. This ensures that they are enqueued transactionally. A scheduled job wakes up and actually tries to execute the queued FS commands.
Register a Synchronization with the current Transaction, fire an appropriate event from there. Your TransactionalFileSystemManager observes this event, no during attribute needed I guess.

Logging exceptions in Windows Service with WinRun4j

I have been trying to get a Windows service running from my JAR file. WinRun4j seems to be able to do the job, but I can't get it to work. I am especially finding it quite difficult to debug. I tried several methods for logging (writing to a .txt file, WinRun4j's EventLog class) but I can't seem to generate any output.
The service installs fine (eventually..) and I can start it. It should start a Jetty server that generates an XML file that can be reached over HTTP. The app works for a stand-alone version, just not for the service. The service is started, but as soon as I call the URL it stops without generating an error.
This is my Service class:
package com.some.package;
import org.boris.winrun4j.AbstractService;
import org.boris.winrun4j.ServiceException;
/**
* A basic service.
*/
public class StockService extends AbstractService {
private StockServer srv;
public int serviceMain(String[] args) throws ServiceException {
while (!shutdown) {
try {
Thread.sleep(5000);
} catch(InterruptedException e) {
}
if(srv == null) {
try {
srv = new StockServer();
srv.start();
} catch (Exception e) {
}
}
}
return 0;
}
}
I found out that the service didn't want to start if I started the Jetty server from the serviceMain class. I had to start a new thread. So StockServer extends Thread:
public class StockServer extends Thread {
private Server server;
public void run() {
if (server == null) {
try {
server = new Server(8080);
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath("/example");
StockServlet stockServlet = new StockServlet();
context.addServlet(new ServletHolder(stockServlet), "/stock/*");
server.setHandler(context);
server.setStopAtShutdown(true);
server.start();
server.join();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
}
Since it runs perfectly fine as Java application I just don't know how to get this thing debugged. I hope one of you can point me in the right direction :).
I ended up using the Java Service Wrapper (JSW). This seemed a lot more complex but ended up to be quite easy. It also provides logging by default so I could easily fix the errors. The JSW had problems finding the correct JDK, since JSW is 32bit and I installed JDK1.7 64 bit (and 1.6 32bit). Installing JDK1.7 32bit fixed it. That might have been the problem with WinRun4j as well, but that is something I will never know :).

Secure Nashorn JS Execution

How can I securely execute some user supplied JS code using Java8 Nashorn?
The script extends some computations for some servlet based reports. The app has many different (untrusted) users. The scripts should only be able to access a Java Object and those returned by the defined members. By default the scripts could instantiate any class using Class.forName() (using .getClass() of my supplied object). Is there any way to prohibit access to any java class not explicitly specified by me?
I asked this question on the Nashorn mailing list a while back:
Are there any recommendations for the best way to
restrict the classes that Nashorn scripts can create to a whitelist?
Or is the approach the same as any JSR223 engine (custom classloader
on the ScriptEngineManager constructor)?
And got this answer from one of the Nashorn devs:
Hi,
Nashorn already filters classes - only public classes of non-sensitive packages (packages listed in package.access security
property aka 'sensitive'). Package access check is done from a
no-permissions context. i.e., whatever package that can be accessed
from a no-permissions class are only allowed.
Nashorn filters Java reflective and jsr292 access - unless script has RuntimePermission("nashorn.JavaReflection"), the script wont be
able to do reflection.
The above two require running with SecurityManager enabled. Under no security manager, the above filtering won't apply.
You could remove global Java.type function and Packages object (+ com,edu,java,javafx,javax,org,JavaImporter) in global scope and/or
replace those with whatever filtering functions that you implement.
Because, these are the only entry points to Java access from script,
customizing these functions => filtering Java access from scripts.
There is an undocumented option (right now used only to run test262 tests) "--no-java" of nashorn shell that does the above for you. i.e.,
Nashorn won't initialize Java hooks in global scope.
JSR223 does not provide any standards based hook to pass a custom class loader. This may have to be addressed in a (possible) future
update of jsr223.
Hope this helps,
-Sundar
Added in 1.8u40, you can use the ClassFilter to restrict what classes the engine can use.
Here is an example from the Oracle documentation:
import javax.script.ScriptEngine;
import jdk.nashorn.api.scripting.ClassFilter;
import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
public class MyClassFilterTest {
class MyCF implements ClassFilter {
#Override
public boolean exposeToScripts(String s) {
if (s.compareTo("java.io.File") == 0) return false;
return true;
}
}
public void testClassFilter() {
final String script =
"print(java.lang.System.getProperty(\"java.home\"));" +
"print(\"Create file variable\");" +
"var File = Java.type(\"java.io.File\");";
NashornScriptEngineFactory factory = new NashornScriptEngineFactory();
ScriptEngine engine = factory.getScriptEngine(
new MyClassFilterTest.MyCF());
try {
engine.eval(script);
} catch (Exception e) {
System.out.println("Exception caught: " + e.toString());
}
}
public static void main(String[] args) {
MyClassFilterTest myApp = new MyClassFilterTest();
myApp.testClassFilter();
}
}
This example prints the following:
C:\Java\jre8
Create file variable
Exception caught: java.lang.RuntimeException: java.lang.ClassNotFoundException:
java.io.File
I've researched ways of allowing users to write a simple script in a sandbox that is allowed access to some basic objects provided by my application (in the same way Google Apps Script works). My conclusion was that this is easier/better documented with Rhino than with Nashorn. You can:
Define a class-shutter to avoid access to other classes: http://codeutopia.net/blog/2009/01/02/sandboxing-rhino-in-java/
Limit the number of instructions to avoid endess-loops with observeInstructionCount: http://www-archive.mozilla.org/rhino/apidocs/org/mozilla/javascript/ContextFactory.html
However be warned that with untrusted users this is not enough, because they can still (by accident or on purpose) allocate a hugh amount of memory, causing your JVM to throw an OutOfMemoryError. I have not found a safe solution to this last point yet.
You can quite easily create a ClassFilter which allows fine-grained control of which Java classes are available in JavaScript.
Following the example from the Oracle Nashorn Docs:
class MyCF implements ClassFilter {
#Override
public boolean exposeToScripts(String s) {
if (s.compareTo("java.io.File") == 0) return false;
return true;
}
}
I have wrapped this an a few other measures in a small library today: Nashorn Sandbox (on GitHub). Enjoy!
So far as I can tell, you can't sandbox Nashorn. An untrusted user can execute the "Additional Nashorn Built-In Functions" listed here:
https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html
which include "quit()". I tested it; it exits the JVM entirely.
(As an aside, in my setup the global objects, $ENV, $ARG, did not work, which is good.)
If I'm wrong about this, someone please leave a comment.
The best way to secure a JS execution in Nashorn is to enable the SecurityManager and let Nashorn deny the critical operations.
In addition you can create a monitoring class that check the script execution time and memory in order to avoid infinite loops and outOfMemory.
In case you run it in a restricted environment without possibility to setup the SecurityManager, you can think to use the Nashorn ClassFilter to deny all/partial access to the Java classes. In addition to that you must overwrite all the critical JS functions (like quit() etc.).
Have a look at this function that manage all this aspects (except memory management):
public static Object javascriptSafeEval(HashMap<String, Object> parameters, String algorithm, boolean enableSecurityManager, boolean disableCriticalJSFunctions, boolean disableLoadJSFunctions, boolean defaultDenyJavaClasses, List<String> javaClassesExceptionList, int maxAllowedExecTimeInSeconds) throws Exception {
System.setProperty("java.net.useSystemProxies", "true");
Policy originalPolicy = null;
if(enableSecurityManager) {
ProtectionDomain currentProtectionDomain = this.getClass().getProtectionDomain();
originalPolicy = Policy.getPolicy();
final Policy orinalPolicyFinal = originalPolicy;
Policy.setPolicy(new Policy() {
#Override
public boolean implies(ProtectionDomain domain, Permission permission) {
if(domain.equals(currentProtectionDomain))
return true;
return orinalPolicyFinal.implies(domain, permission);
}
});
}
try {
SecurityManager originalSecurityManager = null;
if(enableSecurityManager) {
originalSecurityManager = System.getSecurityManager();
System.setSecurityManager(new SecurityManager() {
//allow only the opening of a socket connection (required by the JS function load())
#Override
public void checkConnect(String host, int port, Object context) {}
#Override
public void checkConnect(String host, int port) {}
});
}
try {
ScriptEngine engineReflex = null;
try{
Class<?> nashornScriptEngineFactoryClass = Class.forName("jdk.nashorn.api.scripting.NashornScriptEngineFactory");
Class<?> classFilterClass = Class.forName("jdk.nashorn.api.scripting.ClassFilter");
engineReflex = (ScriptEngine)nashornScriptEngineFactoryClass.getDeclaredMethod("getScriptEngine", new Class[]{Class.forName("jdk.nashorn.api.scripting.ClassFilter")}).invoke(nashornScriptEngineFactoryClass.newInstance(), Proxy.newProxyInstance(classFilterClass.getClassLoader(), new Class[]{classFilterClass}, new InvocationHandler() {
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
if(method.getName().equals("exposeToScripts")) {
if(javaClassesExceptionList != null && javaClassesExceptionList.contains(args[0]))
return defaultDenyJavaClasses;
return !defaultDenyJavaClasses;
}
throw new RuntimeException("no method found");
}
}));
/*
engine = new jdk.nashorn.api.scripting.NashornScriptEngineFactory().getScriptEngine(new jdk.nashorn.api.scripting.ClassFilter() {
#Override
public boolean exposeToScripts(String arg0) {
...
}
});
*/
}catch(Exception ex) {
throw new Exception("Impossible to initialize the Nashorn Engine: " + ex.getMessage());
}
final ScriptEngine engine = engineReflex;
if(parameters != null)
for(Entry<String, Object> entry : parameters.entrySet())
engine.put(entry.getKey(), entry.getValue());
if(disableCriticalJSFunctions)
engine.eval("quit=function(){throw 'quit() not allowed';};exit=function(){throw 'exit() not allowed';};print=function(){throw 'print() not allowed';};echo=function(){throw 'echo() not allowed';};readFully=function(){throw 'readFully() not allowed';};readLine=function(){throw 'readLine() not allowed';};$ARG=null;$ENV=null;$EXEC=null;$OPTIONS=null;$OUT=null;$ERR=null;$EXIT=null;");
if(disableLoadJSFunctions)
engine.eval("load=function(){throw 'load() not allowed';};loadWithNewGlobal=function(){throw 'loadWithNewGlobal() not allowed';};");
//nashorn-polyfill.js
engine.eval("var global=this;var window=this;var process={env:{}};var console={};console.debug=print;console.log=print;console.warn=print;console.error=print;");
class ScriptMonitor{
public Object scriptResult = null;
private boolean stop = false;
Object lock = new Object();
#SuppressWarnings("deprecation")
public void startAndWait(Thread threadToMonitor, int secondsToWait) {
threadToMonitor.start();
synchronized (lock) {
if(!stop) {
try {
if(secondsToWait<1)
lock.wait();
else
lock.wait(1000*secondsToWait);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
if(!stop) {
threadToMonitor.interrupt();
threadToMonitor.stop();
throw new RuntimeException("Javascript forced to termination: Execution time bigger then " + secondsToWait + " seconds");
}
}
public void stop() {
synchronized (lock) {
stop = true;
lock.notifyAll();
}
}
}
final ScriptMonitor scriptMonitor = new ScriptMonitor();
scriptMonitor.startAndWait(new Thread(new Runnable() {
#Override
public void run() {
try {
scriptMonitor.scriptResult = engine.eval(algorithm);
} catch (ScriptException e) {
throw new RuntimeException(e);
} finally {
scriptMonitor.stop();
}
}
}), maxAllowedExecTimeInSeconds);
Object ret = scriptMonitor.scriptResult;
return ret;
} finally {
if(enableSecurityManager)
System.setSecurityManager(originalSecurityManager);
}
} finally {
if(enableSecurityManager)
Policy.setPolicy(originalPolicy);
}
}
The function currently use the deprecated Thread stop(). An improvement can be execute the JS not in a Thread but in a separate Process.
PS: here Nashorn is loaded through reflexion but the equivalent Java code is also provided in the comments
I'd say overriding the supplied class's classloader is easiest way to control access to classes.
(Disclaimer: I'm not really familiar with newer Java, so this answer may be old-school/obsolete)
An external sandbox library can be used if you don't want to implement your own ClassLoader & SecurityManager (that's the only way of sandboxing for now).
I've tried "The Java Sandbox" (http://blog.datenwerke.net/p/the-java-sandbox.html) although it's a bit rough around the edges, but it works.
Without the use of Security Manager it is not possible to securely execute JavaScript on Nashorn.
In all releases of Oracle Hotspot that included Nashorn one can write JavaScript that will execute any Java/JavaScript code on this JVM.
As of January 2019, Oracle Security Team insist that use of Security Manager is mandatory.
One of the problems is already discussed in https://github.com/javadelight/delight-nashorn-sandbox/issues/73

How to setup a logger in an OSGi Enterprise Application on WebSphere?

I'm quite confused on how to properly setup a logger in an Java EE OSGi environment. Here are my requirements:
Only 1 log file per EBA (grouping of bundles)
Multiple log files per application server (due to multiple EBAs)
Do not want to perform ClassLoader magic (its fine if a library I use does this, I just don't want to have to write it)
Must rotate the log file at the end of the day and only maintain 7 log files at once
Preferably does not require creating a logging bundle per EBA. No other developer will buy in if I tell them to write their own logging interop for each application.
Must work with WebSphere Application Server 8.5.5
First I tried to use SLF4j on top of log4j like our other Java EE applications do, however nothing could find my log4j.properties. I tried variations of importing SLF4j, and also had issues where loading it in 1 bundle prevented it from loading in another.
Next I looked into PAX logger, but it appears to log globally, not per EBA.
Attempting to use the OSGi LogService prevents my bundle from deploying to WebSphere, plus I'm not sure how I could get it to meet my requirements anyway.
I'm at the point where the only option I can see is to write my own bundle that keeps a registry for bundle -> log file (using FrameworkUtil.getBundle on the client class) and implement a full logging framework within it. If that has classloader isolation issues then possibly push to an EJB to do the actual logging. I'm really hoping that's not my only solution.
Can anyone point me to some documentation that will help me out??
Thank you!
People are in general confused about the Log Service ... The Log Service does not store any logs, it just acts as a dispatcher. I understand the confusion since the Log Service is mandated to have a small buffer for the initial start up and provides an API to get the buffer.
For what you want, you should add a Log Listener with the Log Reader services. What you want is quite easy with Declarative Services. This is a component that implements your requirements :
#Component(provide = {}, immediate = true) public class Logger extends Thread
implements LogListener {
final BlockingQueue<LogEntry> queue = new ArrayBlockingQueue<LogEntry>(1000);
File root;
#Reference void setLR(LogReaderService lr) {
lr.addLogListener(this);
}
#Activate void activate(BundleContext context, Map<String,Object> props) {
if ( props.containsKey("root"))
root = new File((String) props.get("root"));
else
root = context.getDataFile("logs");
root.mkdirs();
start();
}
#Deactivate void deactivate() {
interrupt();
}
#Override public void logged(LogEntry entry) {
queue.offer(entry); // ignore full silently
}
public void run() {
while (true)
try {
LogEntry entry = queue.take();
File file = getPath(entry);
if (file.isFile()) {
long days = TimeUnit.MILLISECONDS.toDays(System.currentTimeMillis()
- file.lastModified());
if (days > 2) file.delete();
}
try (OutputStream raf = new FileOutputStream(file, true)) {
String s = String.format("%tT [%03d] %s%n", entry.getTime(), entry
.getBundle().getBundleId(), entry.getMessage());
raf.write(s.getBytes("UTF-8"));
}
} catch (InterruptedException ie) {
return;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private File getPath(LogEntry entry) {
long rollover = TimeUnit.MILLISECONDS.toDays(System.currentTimeMillis()) % 7;
String eba = "eba"; // fill in eba from entry.getBundle()?
return new File(root, "log-" + eba + "-" + rollover + ".msg");
}
}
This could of course be done a bit more efficient, but that is left as an exercise.

H2 on Tomcat shutdown hanging, memory leak when using Tomcat Datasource

This is related to these 2 posts:
What is the proper way to close H2?
Tomcat doesn't stop. How can I debug this?
Basically H2 keeps a lock on the database, even when all connections are closed, and so when stopping Tomcat it hangs waiting on a thread, the process is still running.
The only way I managed to get H2 to not lock the database is by issueing the statement SHUTDOWN IMMEDIATELY command (the vanilla or compact did not release the lock).
This is performed in my ServletContextListener class in the contextDestroyed like this (I have omitted comments and log lines):
ServletContext ctx = servletContextEvent.getServletContext();
DataSource closeDS = databaseConnection.getDatasource();
Connection closeConn = null;
PreparedStatement closePS = null;
try {
closeConn = closeDS.getConnection();
closePS = closeConn.prepareStatement("SHUTDOWN IMMEDIATELY");
closePS.execute();
} catch (Exception ex) {
} finally {
if (closePS != null) {
try { closePS.close(); } catch (SQLException ex) {}
}
if (closeConn != null) {
try { closeConn.close(); } catch (SQLException ex) {}
}
}
try {
databaseConnection.close();
databaseConnection = null;
ctx.setAttribute("databaseConnection", null);
} catch(Exception e) {
}
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver driver = drivers.nextElement();
try {
DriverManager.deregisterDriver(driver);
} catch (Exception e) {
}
}
Now the lock is released Tomcat stops (although I still get the severe memory leak messages in the logs) but now I receive also a number of error stacks in the logs thus:
INFO: Illegal access: this web application instance has been stopped already.
Could not load java.lang.ThreadGroup.
The eventual following stack trace is caused by an error thrown
for debugging purposes as well as to attempt to terminate
the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)
at org.h2.engine.DatabaseCloser.reset(DatabaseCloser.java:43)
at org.h2.engine.Database.close(Database.java:1155)
at org.h2.engine.DatabaseCloser.run(DatabaseCloser.java:80)
10-sep-2013 13:31:41 org.apache.catalina.loader.WebappClassLoader loadClass
The question is: how can I shut down the database without causing illegal state exceptions. Is there something wrong in my code to call the shutdown command?
Why is this such an issue with H2? I do not have this issue with JBoss or Websphere where the application also runs using datasources provided by the container.

Categories

Resources