Develop Reference

Java - netty library execution time very big - Java NIO

I am developing a Java application that reads data from a Redis Database, I use Lettuce library to connect to Redis which in turn uses 'Netty' library to communicate with Redis
I suspect that the execution time of my application is greater than expected, so a conducted a profiling experiment using JProfiler, I was surprised that a FastThreadLocalRunnable takes a significant portion of the execution time with no justification as the tree shows no function calls taking time:
So, is it a bug in Lettuce library?, or is it a problem in the profiler measuring the execution time?
Any help is appreciated
Edit:
Thanks to Ingo's answer I can now expand the tree but it turns out that the java NIO is consuming my processor:
Any idea?

The call tree in JProfiler only shows classes that are included in the call tree filters that you define in the profiling settings:
By default, this excludes a lot of common frameworks and libraries so that you can get started without configuring anything. It is better if you delete these filters and add your own profiled packages here.
In addition to the profiled classes, JProfiler shows the thread entry point even it is not a profiled class, such as io.netty.util.concurrent.FastThreadLocalRunnable. Also, the first call into non-profiled classes is always shown at any level in the call tree.
In your case there are call chains to non-profiled classes below io.netty.util.concurrent.FastThreadLocalRunnable that never call a profiled class. They could belong to some framework or to some part of your code that is not included in the profiled classes. This time has to go somewhere, so it is attributed to the io.netty.util.concurrent.FastThreadLocalRunnable node.
An easy way to check is to disable filtering in the profiling settings, then you see all classes.
More information about call tree filters can be found at
https://www.ej-technologies.com/resources/jprofiler/help/doc/main/methodCallRecording.html

How to use the Java Instrumentation API to reload classes when they change on the the file system?

I don't want to use the URL Classloader to load classes.
I want to implement this myself.
I don't want to use a solution like JRebel (although it's great).
I've got prior experience of JavaAssist, bytecode generation, implementing javaagent class transformers etc.
I would like to write a javaagent which hooks into the classloader or defines it's own system classloader.
I'll store the class files in an in memory cache, and for particular files, periodically reload them from disk.
I'd prefer to do this in a way which doesn't involve continuously polling the file system and manually invalidating specific classes. I'd much rather intercept class loading events.
I last messed around with this stuff 4 years ago, and I'm sure, although my memory may deceive me that it was possible to do, but 8 hours of searching google doesn't present an obvious solution beyond building a patched JVM.
Is this actually possible?
I've created a stub implementation at https://github.com/packetops/poc_agent if anyone's interested in a simple example of javaagent use.
update
Just found this post - I may have been using the wrong approach, I'll investigate further.

It depends on what you want to do. If you want to reload your classes and define new ones, then you are fine with implementing your own classloader, as you already found.
If you want to replace existing classes, things become more "envolved". You can do this by implementing your own tiny Java agent. See the Java documentation, how to do this: http://docs.oracle.com/javase/7/docs/api/java/lang/instrument/package-summary.html
With the instrumentation mechanism you can not freely redefine classes, quote from Instrumentation.redefineClass:
The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields or methods, change the signatures of methods, or change inheritance. These restrictions maybe be lifted in future versions. The class file bytes are not checked, verified and installed until after the transformations have been applied, if the resultant bytes are in error this method will throw an exception.
If you want to do more, you need to load it again. This can be done under the same name, by using a different classloader. The previous class definition will be unloaded, if no one else is using it any more. So, you need to reload any class that uses your previous class also. Utlimatly, you end up reinventing something like OSGi. Take a look at: Unloading classes in java?

How to refactor procedural start-up code?

I have a class (Android Activity) which handles start-up of my application. The application has some pretty complex start-up rules. Right now it looks like a bunch of spaghetti and I'm looking for strategies for refactoring it.
It's honestly such a mess I'm having problems hacking it down to provides pseudo code. In general there are some rules for start-up that are basically codified in logic:
Steps:
Check for error on last exit and flush local cache if necessary
Download settings file
Parse settings and save settings to local native format
Using the values in settings, do a bunch of 'house keeping'
Using a value in settings, download core data component A
Parse component A and load up local cache
During this logic, its also updating the user interface. All of this is handled in a zig-zagging, single monolithic class. Its very long, its got a bunch of dependencies, the logic is very hard to follow and it seems to touch way too many parts of the application.
Is there a strategy or framework that can be used to break up procedural start-up code?

Hmmm. Based on your steps, I see various different "concerns":
Reading and saving settings.
Downloading settings and components (not sure what a "component" is here) from the server.
Reading and instantiating components.
Flush and read cache.
Housekeeping (not really sure what this all entails).
UI updates (not really sure what this requires either).
You might try splitting up the code into various objects along the lines of the above, for example:
SettingsReader
ServerCommunicationManager (?)
ComponentReader
Cache
Not sure about 5 and 6, since I don't have much to go on there.
Regarding frameworks, well, there are various ones such as the previously mentioned Roboguice, that can help with dependency injection. Those may come in handy, or it may be easier just to do this by hand. I think that before you consider dependency injection, though, you need to untangle the code. All that dependency injection frameworks do is to initialize your objects for you -- you have to make sure that the objects make sense first.

Without any more details, the only suggestion that I can think of is to group the various steps behind well structured functions which do one thing and one thing only.
Your 6 steps look to be a good start for the 6 functions your init function should have. If #2 was synchronous (I doubt it), I would merge #2, #3 into a getSettings function.

Use of AspectJ for debugging Enterprise Java applications

The idea is to utilize AOP for designing applications/tools to debug/view execution flow of an application at runtime. To begin with, a simple data(state) dump at the start and end of method invocation will do the necessary data collection.
The target is not application developers but high level business analyst or high level support people for whom a execution flow could prove helpful. The runtime application flow can also be useful in reducing the learning curve of an application for new developers especially in configuration loaded systems.
I wanted to know if there already exists such tools/applications which could be used. Or better, if this makes sense, then is there a better way to achieve this.

You could start with Spring Insight (http://www.springsource.org/insight) and add your own plugins to collect data appropriate for business analysts/support staff. If that doesn't meet needs, you can write your own custom aspects. It is not that hard.

You could write your own aspects, as suggested by ramnivas, but to prepare for the requests from the users, you may want to just have the aspects compiled into the application, so that you don't have to take a hit at run-time, and then they could just select which execution flows or method groups they are interested in, and you just call the server and set some variable to give them the information desired.
Writing the aspects is easy, but to limit recompiling, you may want to get an idea what the users will want, for example, if they want to have a log of every call made from the time a webservice is called until it gets to the database, then you can build that in, but it would be easier to know this up-front.
Otherwise the aspect does nothing, if the variable is not set, and perhaps unset the variable when finished.
You could also have where they can pick which type of logging and for which user, which may lead to more useful information.

How can I preload web app classes in the JVM on start up?

In our web apps the first load of some pages takes a small but noticeable extra about of time due to class loading. Does anyone have any clever ways of preloading web app classes in the JVM on start up?
Update: What we do now is store a bunch (700) of full class names in a db table. We read the table at startup and do Class.forName(). It works fine but I thought there might be a more clever approach. We determined the 700 classes that were referenced at startup by using a profiler.

Well, it's not very clever as it's part of the specification, but you can have your servlets be started when the Web App is started by adding the load-on-startup element to the servlet definition in web.xml:
<servlet>
<description>....</description>
<display-name>....</display-name>
<servlet-name>....</servlet-name>
<servlet-class>....t</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
If you do this for a representative set of services -- or perhaps just one servlet that can preload everything needed -- then you'll achieve your goal.
If this isn't sufficient, if you want to load classes from JAR files, for example, without actually initializing them in a meaningful way, and if you know where your JAR files are or can figure out where your JAR files are, then you can use something like the code at this JCP forum post "List classes in package" or some of the later posts in that thread. From the list of classes, you can get the Class objects, which will help load the classes without actually having to instantiate an instance.

You might try writing static initializer code in a single dummy class and then load that class at startup. Its static initializer would create a few key objects that cause other classes to be loaded (and you could do this recursively to improve code modularity). I'd bet this is much shorter and simpler (and you have no DB issues to worry about).
A better approach probably is to write a servlet that on startup hits a few of your slow-loading pages and throws away the results. This forces class loads. Loading these pages many times each will increase the amount just-in-time compilation done (both speeding up your code and getting JIT compilation costs out of the way). There are other advantages too: this is a "power-on self-test", it causes one-time startup tasks to be done, and it may also prime various caches to some noticable degree.

Class.forName() is about the only thing I can think of. I'd certainly be interested in hearing more clever alternatives.
Another option would be to select a set of URLs and run a script on startup to hit those URLs.

The Class.forName inside of a thread would potentially speed things up. Start with the ones that are on the first page likely to be hit.
Threading it should make the startup quicker (it'll return earlier) and since you start loading the more likely classes first they will hopefully be loaded by the time the page gets hit. For the others same deal, hopefully loaded before the page gets hit.
You could also take it further and start a thread for each group of classes (those classes needed on a given page would be a group). Which might speed things up since you could parallelize the reading from disk (but that might also slow it down).
This doesn't grantee that things won't slow down on the first load of a page, but it is worth investigating.

You could always use a third party jsp compiler, such as Jasper, to precompile your JSP pages before you deploy them.

May be you can try converting the 700 classes to a jar file and apply some methods for preloading,probably by putting the jar in boot class path (I am not sure if it works ..but just giving a hint ).