I have a Tomcat 6 server which is hosting a portlet container (in a war) and several portlets (each one packaged in its own war). We have a requirement to share data between these portlets, and due to the version of the portlet container we are using, the only way that we can see to do this is to place a jar with our logic into the endorsed folder to ensure that the same instance is seen by the different wars (each one has its own classloader).
This jar makes use of the various caching functions of Google Guava and has been proven to work in unit tests and within the portlet container. However, we have found ourselves faced with a clash of different versions of Guava due to the endorsed classloader taking preference over the subsequent classloaders. The portlet container uses an early version of Guava and our portlets use our internal framework which uses a more recent version!
If we write our endorsed jar to use the same version of Guava as the portlet container then the framework boots but the portlets won't start as the API is different.
If we write our endorsed jar to use the same version of Guava as our own framework, then the portlet framework won't start because the API is different, so our portlets never load.
The approach we are trying at the moment is copying the Guava source code into our own jar under a new package - this has been done in various open source projects - for example the class com.google.common.base.Ascii would be repackaged com.mycompany.com.google.common.base.Ascii. This works, though it means recompiling Guava each time we make a new version of our endorsed jar.
We would prefer being able to automate the generation of this endorsed jar, and having seen documentation for using ProGuard with Guava in an Android context, have a feeling that Proguard might be able to help.
Is ProGuard capable of repackaging a jar dependency inside another jar, maintaining a single internal package hierarchy? Could the -flattenpackagehierarchy option do this?
Another option would be to use JarJar to repackage Guava so you don't have to recompile it from source every time.
Related
I'm currently working on an ebanking platform, so out customers are banks. To extend this platform, we develop our own 'xDK' (development kit) for 3rd party developers (usually the banks themselves).
When xDK is used as a dependency (via maven or gradle), it brings along a lot of transitive dependencies in order to work (~25MB). I was trying to think of solutions to make the dependency a bit lighter to use (given that it needs all of its dependencies) which in turn will promote having smaller, more focused services (not exactly micro-services but at least a step closer).
The current situation's benefit is that every service/project can use its own version of xDK and it doesn't have to update until it needs to. The problem is that it doesn't scale. If we assume 100 WAR files having xDK as a dependency, we create a 2.5GB overhead on the application server (even if they all use the same version).
I'll list two options I was thinking of, but I'd like to know if there are better solutions for this problem. Feel free to ask for more info. Thanks in advance.
Similar to JavaEE components (JPA, JAX-RS, ...), we'll have an 'api' dependency and the implementation. The projects will only declare the 'api' as a provided dependency while the implementation will be provided like so:
JBoss module
I haven't worked with other application servers. We (and our customers) only use JBoss EAP, so this might be a JBoss specific solution. We can create a JBoss module for xDK and then make every deployment depend on it via the JBoss deployment descriptor. The benefit is that we get rid of the multiple copies of the library, but we lose on version flexibility. This would mean that there needs to be some kind of governance on which version of xDK you code against in your service. Also, every time there is a breaking change, we'd need to update all services if we want to update the JBoss module to the latest version.
Bundle in an EAR
EARs allow multiple WAR files in them and also jars as libs. xDK will be an EAR dependency. Again, we have the same pros and cons as the previous solution. This solution is JBoss independent. However, it needs an extra build step to collect all the projects and bundle them, which might be annoying for out customers if they need to bundle their own services.
How about using the maven dependency scope of provided to declare that for the individual war files the jar file is provided outside of the war file, and then have another mechanism to inject the shared jar file into the application server?
c.f. https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
I've downloaded the latest (9.3.7.v20160115) Jetty from here.
Accourding to the documentation, the HTTP/2 has 4 sub-projects, one is http2-client.
I'm unable to find it. The only jars I have in http2 are:
http2-common-9.3.7.v20160115.jar
http2-hpack-9.3.7.v20160115.jar
http2-server-9.3.7.v20160115.jar
What has happened with http2-client?
All Jetty components are available on Global Maven Repository system.
Here's the components belonging to org.eclipse.jetty.http2, along with the http2-client you are looking for.
Since there is nothing in the jetty-distribution that uses the http2-client (and associated alpn-client), and there is no way to use the http2-client through the server / webapp classloader isolation, its highly unlikely that it will show up on the jetty-distribution.
If you intend to use it standalone (not from a java web/app server), then use the components from maven central.
If your intention is to use it from a jetty server instance, you'll want to copy the components into place in your WEB-INF/lib directory for your webapp (be aware that you should have your http2-client and jetty server versions in lockstep if you attempt to do this, as you can potentially share many utility/io/http classes from jetty server classloader itself)
According to Bugzilla and GitHub issue this is a bug.
We have a web application made in Java, which uses struts2, spring and JasperReport. This application runs on glassfish 4.0.
The libraries of the application are in the WEB-INF/lib folder, and also in glassfish are installed 4 more than uses the same libraries.
Glassfish is configured to use 1024mb for heapspace and 512m for permgen, and the most of the memory consumption when i use libraries per application is in the struts actions and spring aop classes (using netbeans profiler).
The problem we are having is the amount of memory consumed by having libraries in the classloader per application because is to high and generates PermGen errors and we have also noticed that the application run slower with more users.
because of that we try to use shared-libraries, put it in domain1/lib folder and found that with a single deployed application the load time and memory consumption is much lower, and the application works faster in general. But when we deploy the rest of the applications on the server only the first application loaded works well and the rest has errors when we calls struts2 actions.
We believe that is because each application has slightly different settings on struts2 and log4j.
We have also tried to put only certain libraries on glassfish and leaving only struts2 in the application but it shows InvocationTargetException errors because all libraries depend the lib from apache-common and it dont matter if we put those lib on one place or another. Also if we put it in both places the application don’t start.
there any special settings or best practices for using shared-libraries?
Is there a way to use shared-libraries but load settings per application? or we have to change the settings to make them all the same?
Is there any special settings or best practices for using shared-libraries? Is there a way to use shared-libraries but load settings per application? or we have to change the settings to make them all the same?
These are actually interesting questions... I don't use GlassFish but, according to the documentation :
Application-Specific Class Loading
[...]
You can specify module- or application-specific library classes [...] Use the asadmin deploy command with the --libraries option and specify comma-separated paths
[...]
Circumventing Class Loader Isolation
Since each application or individually deployed module class loader universe is isolated, an application or module cannot load classes from another application or module. This prevents two similarly named classes in different applications or modules from interfering with each other.
To circumvent this limitation for libraries, utility classes, or individually deployed modules accessed by more than one application, you can include the relevant path to the required classes in one of these ways:
Using the Common Class Loader
Sharing Libraries Across a Cluster
Packaging the Client JAR for One Application in Another Application
Using the Common Class Loader
To use the Common class loader, copy the JAR files into the domain-dir/lib or as-install/lib directory or copy the .class files (and other needed files, such as .properties files) into the domain-dir/lib/classes directory, then restart the server.
Using the Common class loader makes an application or module accessible to all applications or modules deployed on servers that share the same configuration. However, this accessibility does not extend to application clients. For more information, see Using Libraries with Application Clients. [...]
Then I would try:
Solution 1
put all the libraries except Struts2 jars under domain1/lib ,
put only Struts2 jars under domain1/lib/applibs,
then run
$ asadmin deploy --libraries struts2-core-2.3.15.2.jar FooApp1.war
$ asadmin deploy --libraries struts2-core-2.3.15.2.jar FooApp2.war
To isolate Struts2 libraries classloading while keeping the rest under Common Classloader's control.
Solution 2
put all the libraries except Struts2 jars under domain1/lib ,
put only Struts2 jars under domain1/lib/applibs, in different copies with different names, eg appending the _appname at the jar names
then run
$ asadmin deploy --libraries struts2-core-2.3.15.2_FooApp1.jar FooApp1.war
$ asadmin deploy --libraries struts2-core-2.3.15.2_FooApp2.jar FooApp2.war
To prevent sharing of the libraries by istantiating (mock) different versions of them.
Hope that helps, let me know if some of the above works.
You can try to create what is known as a skinny WAR. Pack all your WARs inside an EAR and move all the common JARs from WEB-INF/lib to the lib/ folder in the EAR (don't forget to set <library-directory> in the application.xml).
I'd bet that placing the libs under lib/ or lib/ext won't resolve your performance issues. You did not write anything about the applications or server settings, like size of application, available Heap and PermGen space, but nonetheless I would recommend to stay with separate libs per app.
If you place the libs in server dirs, they will be shared among all apps. You will loose the option to upgrade only one of your applications to a new framework or to get rid away of any of them. Your deployment will be bound to a specific server architecture.
And you wrote it did not solve your problems, it even may raise new ones.
I would recommend to invest some hours into tuning the server. If it runs with defaults, allocate more PermGen and HeapSpace.
If this does not help, you should analyze in deep what's going wrong. Shared libs might be a solution, but you don't know the problem, yet. IBM offer some cool and free tools to analyze heap dumps, this could be a good starting point.
I came here in search of guidance about installing libraries that are shared among multiple applications or projects. I am deeply disappointed to read that the accepted practice favors installing a copy of every shared library into each project. So, if you have ten Web application, all of which use, e. g., httpcomponents-client, mysql-connector-java, etc., then your installation contains ten copies of each.
This behavior reminds me, painfully, of the way of thinking that motivated me to abandon the mainframe in favor of the PC; the thinking seemed to be "I don't care how many resources my application consumes. In fact, I'd like to be able to brag about what a resource hog it is." Excuse me, please, while I hurl.
The interface exposed by a library is an immutable contract that is not subject to change at the developer's whim.
There is this concept called backwards compatibility. If you break it, you create a new interface.
I know of at least two types of interfaces that adhere to the letter and spirit of these rules.
By far the oldest is the IBM System/370 system libraries. You might have Foo and Foo2, where the latter extends and/or breaks the contract made by the Foo interface in some way that made it incompatible.
From its beginnings in the Bell Labs Unix project, the standard C runtime library has adhered to the above rules.
Though it is much newer, the Microsoft COM interface specification enforces the same rule.
To their credit, Microsoft generally adheres to those rules in the Win32 API, too, although there are a handful of exceptions in that API. To a degree, they went backwards with the .NET Framework, which seems slavishly to follow in the footsteps of the Java environment that it so eagerly seeks to replace.
I've been using libraries since 1978, and my understanding was and is that the goal of putting code into a library was to make it reusable. While maintaining copies of the library code in each application eliminates the need to implement it again for each new project, it severely complicates upgrading, since you now have ten (or more) copies of the library, each of which must be updated.
If libraries adhere to the rule that an interface is an immutable contract, why shouldn't they live in a shared library directory, as do the Unix system libraries that live in its /lib directory, from which everything that runs on the host shares a single copy of the standard C runtime library, Zlib, and so forth.
Color me seriously disappointed.
I wonder why Java webapps specifications imposed a specific directory for dependencies: /WEB-INF/lib.
Indeed, why don't use classic /META-INF/MANIFEST.MF file?
Someone will tell that webapps are so more secure because libs are physically integrated in WAR file.
But, if we think so, we could wonder :
Why for a simple Java application (simple JAR), there's no special directory for dependencies? It could be also more secure because no risk to have a wrong classpath (modified improperly for instance) in manifest file.
Other will tell that webapp is designed to be portable, so benefit of /WEB-INF/lib is to not worry about links dependencies.
I'm curious to know your opinions on the subject.
A jar is a standard library. For normal utility jars its probably better to keep them separate. If a library has a dependency on a different library and you need that dependent library more then once you would waste space, memory and maybe incorporte problems to due different library versions.
The web application however is an application bundle. You have to make sure that everythings works so you add the required libraries.
Because the WEB-INF/lib makes for a very easy, self contained package of libraries and simplifies the entire deploy for the most common use cases.
/WEB-INF/web.xml
/WEB-INF/lib/utils.jar
/WEB-INF/classes/com/example/Servlet.class
/page.jsp
That's a full boat WAR right there, and with Servlet 3.0, the web.xml is basically empty. Simple layout, trivial to create, and a stand alone artifact to work with in the end.
I have a Java project that expects external modules to be registered with it. These modules:
Implement a particular interface in the main project
Are packaged into a uni-jar (along with any dependencies)
Contain some human-readable meta-information (like the module name).
My main project needs to be able to load at runtime (e.g. using its own classloader) any of these external modules. My question is: what's the best way of registering these modules with the main project (I'd prefer to keep this vanilla Java, and not use any third-party frameworks/libraries for this isolated issue)?
My current solution is to keep a single .properties file in the main project with key=name, value=class |delimiter| human-readable-name (or coordinate two .properties files in order to avoid the delimiter parsing). At runtime, the main project loads in the .properties file and uses any entries it finds to drive the classloader.
This feels hokey to me. Is there a better way to this?
The standard approach in Java is to define a Service Provider.
Let all module express their metadata via a standard xml file. Call it "my-module-data.xml".
On your main container startup it looks for a classpath*:my-module-data.xml" (which can have a FrontController class) and delegates to the individual modules FrontController class to do whatever it wants :)
Also google for Spring-OSGI and their doco can be helpful here.
Expanding on #ZZ Coder...
The Service Provider pattern mentioned, and used internally within the JDK is now a little more formalized in JDK 6 with ServiceLoader. The concept is further expanded up by the Netbeans Lookup API.
The underlying infrastructure is identical. That is, both API use the same artifacts, the same way. The NetBeans version is just a more flexible and robust API (allowing alternative lookup services, for example, as well as the default one).
Of course, it would be remiss to not mention the dominant, more "heavyweight" standards of EJB, Spring, and OSGi.