I'm working on a JSR-303 validation framework for GWT. Some of you may have heard of it even though it is a small project. Here is gwt-validation.
In the old days (v1.0) it used a marker interface for each class and each class had metadata generated separately. This was bad because it was not part of the JSR-303 standard and we moved on to the next idea.
In version 2.0 it scans the classpath at runtime using Reflections. This is great. The downside is that it doesn't seem to be able to work inside of containerized environments or those with special restrictions.
This is probably my fault, look at the following code:
//this little snippet goes through the classpath urls and ommits jars that are on the forbidden list.
//this is intended to remove jars from the classpath that we know are not ones that will contain patterns
Set<URL> classPathUrls = ClasspathHelper.forJavaClassPath();
Set<URL> useableUrls = new HashSet<URL>();
for(URL url : classPathUrls) {
boolean use = true;
for(String jar : this.doNotScanJarsInThisList) {
if(url.toString().contains(jar)) {
use = false;
break;
}
}
if(use) {
useableUrls.add(url);
}
use = false;
}
ConfigurationBuilder builder = new ConfigurationBuilder()
.setUrls(useableUrls)
.setScanners( new TypeAnnotationsScanner(),
new FieldAnnotationsScanner(),
new MethodAnnotationsScanner(),
new SubTypesScanner()
)
.useParallelExecutor()
;
this.reflections = new Reflections(builder);
I'm using the filter to remove jars that I know can't have annotations in them that I'm interested in. As I mention this gives a huge speed boost (especially on large classpaths) but the ClasspathHelper.forJavaClassPath() that I'm basing this on probably isn't the best way to go in container environments. (e.g. Tomcat, JBoss)
Is there a better way or at least a way that will work with a container environment and still let my users filter out classes they don't want?
I've looked, some, into how the Hibernate Validation project (the reference implementation for JSR-303) and they appear to at least be using (at least in part) the Annotations Processing in Java 6. This can't be all of the story because that didn't show up until JDK6 and Hibernate Validator is JDK5 compatible. (See: hibernate documentation)
So, as always, there's more to the story.
I've read these threads, for reference:
About Scannotation which has been pretty much replaced by Reflections.
This one but it uses File and I'm not sure what the implications are of that in things like GAE (Google App Engine) or Tomcat.
Another that goes over a lot of the things I've talked about already.
These threads have only helped so much.
I've also read about the annotation processing framework and I must be missing something. It appears to do what I want but then again it appears to only work at compile time which I know isn't what is done by Hibernate Validator. (Can anyone explain how it does scanning? It works on GAE which means it can't use any of the IO packages.)
Further, would this code work better than what I have above?
Set<URL> classPathUrls = ClasspathHelper.forClassLoader(Thread.currentThread().getContextClassLoader());
Could that correctly get the classloader inside of a Tomcat or JBoss container? It seems scan a smaller set of classes and still finish okay.
So, in any case, can anyone help me get pointed in the right direction? Or am I just stuck with what I've got?
You could take a look at Spring's annotation support.
Spring can scan annotations in files (using asm IIRC), and works in and out of a container.
It may not be easy because it goes through Spring's Resource abstraction, but it should be doable to reuse (or extract) the relevant code.
Related
If I wanted to keep a certain Java package free of 3rd party dependencies with ArchUnit, how would I do it?
More specifically I am looking at keeping my domain model in a hexagonal architecture free from spring code. I specified some rules which I believe ought to prevent the model from using spring. However, I am able to use spring annotations like #Component and #Bean without causing a violation.
What I tried so far is
layeredArchitecture().
layer("domain").definedBy(DOMAIN_LAYER).
layer("application").definedBy(APPLICATION_LAYER).
layer("primary-adapters").definedBy(PRIMARY_ADAPTERS).
layer("secondary-adapters").definedBy(SECONDARY_ADAPTERS).
layer("spring").definedBy("org.springframework..")
whereLayer("spring").mayOnlyBeAccessedByLayers("primary-adapters", "secondary-adapters", "application").
because("Domain should be kept spring-free").
check(CLASSES);
As well as
noClasses().that().resideInAPackage(DOMAIN_LAYER).
should().dependOnClassesThat().resideInAPackage("org.springframework..").
check(CLASSES);
noClasses().that().resideInAPackage(DOMAIN_LAYER).
should().accessClassesThat().resideInAPackage("org.springframework..").
check(CLASSES);
Here a code example which executes the tests just fine, although com.example.app.domain.Factory is importing org.springframework....
You can use DescribedPredicate:
void domainSpring() {
DescribedPredicate<JavaAnnotation> springAnnotationPredicate = new DescribedPredicate<JavaAnnotation>("Spring filter") {
#Override
public boolean apply(JavaAnnotation input) {
return input.getType().getPackageName().startsWith("org.springframework");
}
};
classes().that().resideInAPackage(DOMAIN_LAYER).should()
.notBeAnnotatedWith(springAnnotationPredicate).check(CLASSES);
}
You can also go with name matching.
So you don't have to write a custom DescribedPredicate.
ApplicationCoreMustNotDependOnFrameworks = noClasses()
.that().resideInAnyPackage(DOMAIN_LAYER)
.should().dependOnClassesThat().haveNameMatching("org.springframework.")
.orShould().dependOnClassesThat().haveNameMatching("javax.persistence.*")
.because("Domain should be free from Frameworks");
in my case, I wanted an exception to that rule.
I.e., instead of excluding completely Spring, I wanted to accept the classes in the event package (EventHanlder)
so you can replace "org.springframework" with "org.springframework(?!.*event).*" which is a regular expression
I have a (web-)application that needs special configurations and/or extensions based on the customer using the application. I call these additions "plugins" and they are auto discovered by classpath scanning when the application starts. For extensions that is incredibly easy. Let's say I want to have a plugin which adds an API that prints "hello world" when the URL /myplugin/greet is called: I just create a #Controller annotated class with the according #RequestMapping, put this in a myplugin.jar, copy that on the classpath and that's it.
Problems come up when I want to change some defaults and especially if I want to do this multiple times. Let's say my core application has a config like this:
#Configuration
public class CoreConfiguration {
#Bean
public Set<String> availableModules() {
return Collections.singleton("core");
}
}
Now I have two plugins that don't know about each other (but they do know the CoreConfig), but they both want to add themselves to the list of available modules. How would I do that? If I only had a single plugin that wants to override the module list I could override the existing bean from CoreConfiguration, but with two plugins that becomes a problem. What I imagine is something like this:
#Configuration
public class FirstPluginConfiguration {
#Bean
public Set<String> availableModules(Set<String> availableModules) {
Set<String> extendedSet = new HashSet<>(availableModules);
extendedSet.add("FirstPlugin");
return extendedSet;
}
}
Of course a SecondPluginConfiguration would look nearly exactly like this, except that the Set is not extended by "FirstPlugin", but by "SecondPlugin". I tested it to check what would happen and spring will just never call the First/SecondPluginConfiguration "availableModules" methods but it does not show an error either.
Now of course in this case this could easily be solved by using a mutable Set in the CoreConfiguration and then autowiring and extending the set in the other configurations, but for example I also want to be able to add method interceptors to some beans. So for example I might have an interface CrashLogger which has a logCrash(Throwable t) method and in CoreConfiguration a ToFileCrashLogger is created that writes stack traces to files as the name suggests. Now a plugin could say that he also wants to get notified about crashes, for example the plugin wants to ADDITIONALLY send the stacktrace to someone by email. For that matter that plugin could wrap the CrashLogger configured by the CoreConfiguration and fire BOTH. A second plugin could wrap the wrapper again and do something totally different with the stacktrace and still call both of the other CrashLoggers.
The later does sound somewhat like AOP and if I'd just let ALL my beans be proxied (I did not test that) I could autowire them into my plugin configurations, cast them to org.springframework.aop.framework.Advised and then add advices that manipulate behaviour. However it does seem like a huge overkill to generate proxies for each and everyone of my beans just so that that plugin can potentially add one or two advices one one or two beans.
I am having to make use of some existing code. I cannot contact the previous developer. Part of it is a REST application. I can see how it works, but there is a lot of stuff that looks like code duplication. Or there is a tool of some kind which is taking some of the sources and creating articfacts and other sources from that, or it is creating templates, in which code was added. It looks a bit like Jersey but I have not used this in work, so I am not sure. I tried searching for the annotations, but that is not helpful. I may be missing the build files. It was in an eclipse project and I do not seem to have the .project directory.
This project has a lot of partial implementations that got set aside. I am having problems distringushing those from code that should work.
Looking for just "UserEmail", I see:
src/com/gs/dao/user/UserEmailDao.java
src/com/gs/dao/user/UserEmailDaoImpl.java
src/com/gs/service/UserEmailService.java
src/com/gs/service/UserEmailServiceImpl.java
This is not just 4 times the necessary code. Something is driving this structure. But what is it? Any suggestions?
I am seeing code like:
#ApiController("1.0")
public class UserEndpoint extends BaseEndpoint {
Logger logger = Logger.getLogger(UserEndpoint.class);
#Autowired
public UserService userService;
#Autowired
public UserContactService userContactService;
....
The directory structure looks like this:
src/com/gs/cache
src/com/gs/cache/local
src/com/gs/cache/mem
src/com/gs/servlet
src/com/gs/constants
src/com/gs/common
src/com/gs/dao
src/com/gs/dao/service
src/com/gs/dao/service/attr
src/com/gs/dao/user
src/com/gs/dao/user/attr
src/com/gs/dao/comm
src/com/gs/dao/comm/attr
src/com/gs/dao/vg
src/com/gs/dao/vg/attr
src/com/gs/dao/general
src/com/gs/dao/general/attr
src/com/gs/dao/exception
src/com/gs/elasticsearch
src/com/gs/service
src/com/gs/service/utils
src/com/gs/service/helper
src/com/gs/graph
src/com/gs/graph/gateway
src/com/gs/threads
src/com/gs/async
src/com/gs/async/test
src/com/gs/async/handler
src/com/gs/async/impl
src/com/gs/util
src/com/gs/util/xss
src/com/gs/nlp
src/com/gs/exception
src/com/gs/cassandra
src/com/gs/cassandra/dao
src/com/gs/search
src/com/gs/search/service
src/com/gs/rest
src/com/gs/rest/common
src/com/gs/rest/api
src/com/gs/rest/api/test
What the heck is all this stuff? :-)
You're probbaly not going to get one response that answers this. And you may get shut down for the question being too broad, but I will try. First off:
src/com/gs/dao/user/UserEmailDao.java
src/com/gs/dao/user/UserEmailDaoImpl.java
src/com/gs/service/UserEmailService.java
src/com/gs/service/UserEmailServiceImpl.java
That's a pretty common java pattern, You have an email service, and you split that into an interface and an implementation. You might consider it overkill (if the implementation never changes), but some of the tools being used might require interfaces. Same thing with the UserEmailDao data access object. It's pretty normal for java developers to split everything into an interface and an implementation, though it drives people using dynamic languages crazy.
As for what's generating the REST app, you need to track down where the ApiController annotation is coming from. It looks like it might be wrapper around a Spring MVC class. Post the import statement for that annotation, or just follow it your IDE.
Spring is definitely being used to wire the entire app together.
It looks like a pretty typical medium sized java application to me. From the directory structure, I doubt there is any code generation going on.
If there's a pom.xml (maven file) in the application root, that'll tell you everything you need to know about the application.
To summarize the answer shown here Code assist in (jsp /jstl) view for Spring MVC model objects in Eclipse
is not working for me at all, is there a setting that I need to change ?
I have just downloaded the sample spring-mvc-showcase on github, and it doesn't work out of the box on that project (with either 11.1.3 or EAP 12 version both full enterprise editions), see below (I have no idea where it gets formBean from) :
Here is an example from my own project,the screen shot below (bottom frame) shows my controller adding a string attribute to model and returning correct view name. I would then expect shopString to be offered up as autocomplete option when editing that view, however it is not :
sg is a javascript variable - so great it should be there, but where is "shopString" ?.
Is there a setting I need to change or something else I am missing to get this functionality (using 11.1.3 enterprise edition with all the spring plugins).
It is also failing on spring specific variables :
IS their an open source (one of the spring tutorial projects?) where this definitely works ... or is there a setting I need change in my Intellij install (I have tested with a brand new download of the version 12 EAP) ?
One more screenshot below shows all my spring coifg files set up correctly via autodetection, but the code inspections fails ... this is the spring-mvc-showcase project :
There's a standard way to do this, which is not IntelliJ-specific.
<jsp:useBean id="someModel" scope="request" type="foo.bar.SomeModelClass"/>
The type attribute here does not need to be a concrete class, it can be an interface type as well. Typically you'd put these declarations at the start of your JSP/JSPX files, to provide something like a "declaration of model inputs".
Using JSPs in such a declarative way was recommended in the original book on Spring, in fact (Expert One-on-One J2EE Design and Development.). IntelliJ has been providing full code completion for such pages since at least 7 years.
Note that there are additional relevant convenience features in IntelliJ: if an EL variable reference is marked as undefined, you can press Alt-Enter to select a QuickFix, which will insert a declaration like above. It will even try to figure out the actual type, based on the properties you're accessing.
As I understand Spring, there is no declaration for definitions of variables that you may put into your model. The call model.addAttribute() may add an object to the model, either identified by a parameter or automatically generated by the class name of the object.
So imagine the following case where you have more than one method:
#RequestMapping("foo") public String foo(Model model) {
model.addAttribute("model", new Foo());
return new Random().nextBoolean() ? "page" : "someOtherPage";
}
#RequestMapping("bar") public String bar(Model model) {
model.addAttribute("model", new Bar());
model.addAttribute("model", new Foo());
model.addAttribute("model", new Bar());
return new Random().nextBoolean() ? "page" : "someOtherPage";
}
and the JSP would be something like
<c:out ${model.value} />
Since there is no proper mapping of which controllers may under some circumstances forward to which views, nor what exactly lies within the model, your IDE has no real chance to provide you with proper information.
But to support the IDE in suggesting you some useful information, you can use type hints. Therefore, you have to copy the whole reference of an object, e. g. foo and add a JSP comment like:
<%--#elvariable id="foo" type="com.mycompany.SomeObject"--%>
The warning will vanish and the full IDE support is on your side, allowing you to traverse the fields of foo.
One of the nicest things is that the unused getter warnings will vanish, too. You can directly call the show usages action directly from the JSP or the POJO.
This also works with JSF and particularly within JSF components. Pretty neat feature to have this kind of code completion, showing warnings and errors.
Hope that helps you with your switch to Intellij Idea.
Edit: I also reported this finding to a friend wo wrapped the whole thing into a nice blog entry. Maybe you're interested in reading it: open link
This got fixed in the latest release of intellij 122.694
I faced with similar issue when start writing my own interceptor. Problem was that I start using refference in my view resolver configuration
don't use contruction like this
<bean id="internalResourceViewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
<property name="prefix" ref="prefix"/>
<property name="suffix" ref="suffix"/>
</bean>-
I'm trying to integrate Spring in a pretty large application with thousands of classes, and i'm experiencing huge delays starting my container because of component-scanning.
I have already narrowed the number of directories specified in the "base-package", to the minimum in order to reduce the time wasted in scanning irrelevant directories, but the class-path scanning part of initialization still takes about 1-2 mins.
So, is there a way to optimize the scanning process ? I've thought of storing the candidate classes path in a file and make the container then get them from the file instead of scanning the class-path with every startup, but i don't really know where to start or if that is even possible.
Any advice is much appreciated. Thanks in advance.
Edit1: Loading bean definitions form an autogenerated xml file, reduced the Spring bootstrap time to 9~10 secs which confirms that the reflection api used by Spring for the components class-path scanning is the major source of startup delays.
As for generating the xml file here is the code, since it might be helpful for someone with the same issues.
import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintWriter;
import java.util.ArrayList;
public class ConfigurationWriter {
public ArrayList<String> beanDefinitions = new ArrayList<String>();
public ConfigurationWriter() {
// the context loaded with old fashioned way (classpath scanning)
ApplicationContext context = SpringContainerServiceImpl.getInstance().getContext();
String[] tab = context.getBeanDefinitionNames();
for (int i = 0; i < tab.length - 6; i++) {
Class clazz = context.getType(tab[i]);
String scope = context.isPrototype(tab[i]) ? "prototype" : "singleton";
String s = "<bean id=\"" + tab[i] + "\" class=\"" + clazz.getName() + "\" scope=\"" + scope + "\"/>";
beanDefinitions.add(s);
}
// Collections.addAll(beanDefinitions, tab);
}
#SuppressWarnings("restriction")
public void generateConfiguration() throws FileNotFoundException {
File xmlConfig = new File("D:\\dev\\svn\\...\\...\\src\\test\\resources\\springBoost.xml");
PrintWriter printer = new PrintWriter(xmlConfig);
generateHeader(printer);
generateCorpse(printer);
generateTail(printer);
printer.checkError();
}
#SuppressWarnings("restriction")
private void generateCorpse(PrintWriter printer) {
for (String beanPath : beanDefinitions) {
printer.println(beanPath);
}
}
#SuppressWarnings("restriction")
private void generateHeader(PrintWriter printer) {
printer.println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
printer.println("<beans xmlns=\"http://www.springframework.org/schema/beans\"");
printer.println("xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"");
printer.println("xmlns:context=\"http://www.springframework.org/schema/context\"");
printer.println("xsi:schemaLocation=\"");
printer.println("http://www.springframework.org/schema/mvc");
printer.println("http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd");
printer.println("http://www.springframework.org/schema/beans");
printer.println("http://www.springframework.org/schema/beans/spring-beans-3.0.xsd");
printer.println("http://www.springframework.org/schema/context");
printer.println("http://www.springframework.org/schema/context/spring-context-3.0.xsd\"");
printer.println("default-lazy-init=\"true\">");
}
#SuppressWarnings("restriction")
private void generateTail(PrintWriter printer) {
// printer.println("<bean class=\"com.xxx.frmwrk.spring.processors.xxxBeanFactoryPostProcessor\"/>");
printer.println("<bean class=\"com.xxx.frmwrk.spring.processors.xxxPostProcessor\"/>");
printer.println("</beans>");
}
}
Edit 2: With Spring 5 including an important set of optimizations for speeding up the context initialization, It also comes with an interesting and handy feature that enables generating an index of candidate components at compile time : Spring Context Indexer
Question: How many (in %) of the classes in the directories are Spring Beans?
Answer: I'm not really sure (it's a really big project) , but from what i saw i believe it's arround 90 to 100%, since xml and properties files are isolated in separate locations)
If the problem is really the component scan and not the bean initializing process itself (and I highly doubt that), then the only solution I can imagine is to use Spring XML configuration instead of component scan. - (May you can create the XML file automatically).
But if you have many classes and 90% - 100% of them are Beans, then, the reduction of scanned files will have a maximal improvement of 10%-0%.
You should try other ways to speed up your initialization, may using lazy loading or any lazy loading related techniques, or (and that is not a joke) use faster hardware (if it is not a stand alone application).
A easy way to generate the Spring XML is to write a simple spring application that uses the class path scanning like your original application. After all Beans are initialize, it iterates through the Beans in the Spring Context, check if the bean belongs to the important package and write the XML Config for this bean in a file.
Auto discovery of annotated classes currently requires to scan all classes in the specified package(s) and can take a long time, a known problem of the current class loading mechanism.
Java 9 is going to help here with Jigsaw.
From the Java Platform Module System requirements by Mark Reinold, http://openjdk.java.net/projects/jigsaw/spec/reqs/ :
Efficient annotation detection —
It must be possible to identify all of the class files in a module artifact in which a particular annotation is present without actually reading all of the class files. At run time it must be possible to identify all of the classes in a loaded module in which a particular annotation is present without enumerating all of the classes in the module, so long as the annotation was retained for run time. For efficiency it may be necessary to specify that only certain annotations need to be detectable in this manner.
One potential approach is to augment a module’s definition with an index of the annotations that are present in the module, together with an indication of the elements to which each annotation applies. To limit the size of the index, only annotations which themselves are annotated with a new meta-annotation, say #Indexed, would be included.
Not much you can do about the performance there, I guess you aren't concerned about the startup in production environment, but the startup time of your tests*.
Two tips:
Review that your test-appcontext only uses the minimally required components of your app
instead of having a list of component-scan directives, use one, with a comma-separated value like this: base-package="com.package.one,com.package.two..."
I know it is an old question, and as you will see the situation was different at that time, but hopefully it can help others researching this issue as I did.
According to this answer to a different question, The #ComponentScan annotation now supports a lazyInit flag, which should help in reducing start-up time.
https://stackoverflow.com/a/29832836/4266381
Note: Your edit made it sound like switching to XML by itself was the magic. Yet, looking closer at the code, you had default-lazy-init="true". I wonder if that was the true reason.
The only thing that comes in my mind, beside reducing the directories to be scanned, is the use of lazy bean initialization.
May this could help if you have a lot of beans
You could use Spring's Java-based container configuration instead of component scan.
In comparison to XML-based configuration the Java-based container configuration is type-safe.
But first of all you should check whether your component scan paths are specific enough so that they do not include classes of third party libraries.