For some reasons, I encounter sometimes an empty parquet folder of 0 files or an invalid parquet file.
And then, when a loading attempt is done, I'm receiving the normal exception:
org.apache.spark.sql.AnalysisException: Unable to infer schema for Parquet. It must be specified manually.
But how may I detect that specific error from AnalysisException, to allow my program to ask its deletion and recreation?
it won't be:
catch(AnalysisException e) {
if ("Unable to infer schema for Parquet. It must be specified manually.".equals(e.getMessage())) {
...
}
}
but could the
public String getSqlState()
I'm reading on the AnalysisException javadoc provide any solution? Because my error doesn't look to be of SQL kind.
How can I detect my Spark analysis error using the cleanest manner available for?
Sometimes when I run my application it gives me an error that looks like:
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
at com.example.myproject.Author.getBookTitles(Author.java:25)
at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
People have referred to this as a "stack trace". What is a stack trace? What can it tell me about the error that's happening in my program?
About this question - quite often I see a question come through where a novice programmer is "getting an error", and they simply paste their stack trace and some random block of code without understanding what the stack trace is or how they can use it. This question is intended as a reference for novice programmers who might need help understanding the value of a stack trace.
In simple terms, a stack trace is a list of the method calls that the application was in the middle of when an Exception was thrown.
Simple Example
With the example given in the question, we can determine exactly where the exception was thrown in the application. Let's have a look at the stack trace:
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
at com.example.myproject.Author.getBookTitles(Author.java:25)
at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
This is a very simple stack trace. If we start at the beginning of the list of "at ...", we can tell where our error happened. What we're looking for is the topmost method call that is part of our application. In this case, it's:
at com.example.myproject.Book.getTitle(Book.java:16)
To debug this, we can open up Book.java and look at line 16, which is:
15 public String getTitle() {
16 System.out.println(title.toString());
17 return title;
18 }
This would indicate that something (probably title) is null in the above code.
Example with a chain of exceptions
Sometimes applications will catch an Exception and re-throw it as the cause of another Exception. This typically looks like:
34 public void getBookIds(int id) {
35 try {
36 book.getId(id); // this method it throws a NullPointerException on line 22
37 } catch (NullPointerException e) {
38 throw new IllegalStateException("A book has a null property", e)
39 }
40 }
This might give you a stack trace that looks like:
Exception in thread "main" java.lang.IllegalStateException: A book has a null property
at com.example.myproject.Author.getBookIds(Author.java:38)
at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
Caused by: java.lang.NullPointerException
at com.example.myproject.Book.getId(Book.java:22)
at com.example.myproject.Author.getBookIds(Author.java:36)
... 1 more
What's different about this one is the "Caused by". Sometimes exceptions will have multiple "Caused by" sections. For these, you typically want to find the "root cause", which will be one of the lowest "Caused by" sections in the stack trace. In our case, it's:
Caused by: java.lang.NullPointerException <-- root cause
at com.example.myproject.Book.getId(Book.java:22) <-- important line
Again, with this exception we'd want to look at line 22 of Book.java to see what might cause the NullPointerException here.
More daunting example with library code
Usually stack traces are much more complex than the two examples above. Here's an example (it's a long one, but demonstrates several levels of chained exceptions):
javax.servlet.ServletException: Something bad happened
at com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:60)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.example.myproject.ExceptionHandlerFilter.doFilter(ExceptionHandlerFilter.java:28)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.example.myproject.OutputBufferFilter.doFilter(OutputBufferFilter.java:33)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: com.example.myproject.MyProjectServletException
at com.example.myproject.MyServlet.doPost(MyServlet.java:169)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:30)
... 27 more
Caused by: org.hibernate.exception.ConstraintViolationException: could not insert: [com.example.myproject.MyEntity]
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:96)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.id.insert.AbstractSelectingDelegate.performInsert(AbstractSelectingDelegate.java:64)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2329)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2822)
at org.hibernate.action.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:71)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:268)
at org.hibernate.event.def.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:321)
at org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:204)
at org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:130)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:210)
at org.hibernate.event.def.DefaultSaveEventListener.saveWithGeneratedOrRequestedId(DefaultSaveEventListener.java:56)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:195)
at org.hibernate.event.def.DefaultSaveEventListener.performSaveOrUpdate(DefaultSaveEventListener.java:50)
at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:93)
at org.hibernate.impl.SessionImpl.fireSave(SessionImpl.java:705)
at org.hibernate.impl.SessionImpl.save(SessionImpl.java:693)
at org.hibernate.impl.SessionImpl.save(SessionImpl.java:689)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.hibernate.context.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:344)
at $Proxy19.save(Unknown Source)
at com.example.myproject.MyEntityService.save(MyEntityService.java:59) <-- relevant call (see notes below)
at com.example.myproject.MyServlet.doPost(MyServlet.java:164)
... 32 more
Caused by: java.sql.SQLException: Violation of unique constraint MY_ENTITY_UK_1: duplicate value(s) for column(s) MY_COLUMN in statement [...]
at org.hsqldb.jdbc.Util.throwError(Unknown Source)
at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown Source)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)
at org.hibernate.id.insert.AbstractSelectingDelegate.performInsert(AbstractSelectingDelegate.java:57)
... 54 more
In this example, there's a lot more. What we're mostly concerned about is looking for methods that are from our code, which would be anything in the com.example.myproject package. From the second example (above), we'd first want to look down for the root cause, which is:
Caused by: java.sql.SQLException
However, all the method calls under that are library code. So we'll move up to the "Caused by" above it, and in that "Caused by" block, look for the first method call originating from our code, which is:
at com.example.myproject.MyEntityService.save(MyEntityService.java:59)
Like in previous examples, we should look at MyEntityService.java on line 59, because that's where this error originated (this one's a bit obvious what went wrong, since the SQLException states the error, but the debugging procedure is what we're after).
What is a Stacktrace?
A stacktrace is a very helpful debugging tool. It shows the call stack (meaning, the stack of functions that were called up to that point) at the time an uncaught exception was thrown (or the time the stacktrace was generated manually). This is very useful because it doesn't only show you where the error happened, but also how the program ended up in that place of the code.
This leads over to the next question:
What is an Exception?
An Exception is what the runtime environment uses to tell you that an error occurred. Popular examples are NullPointerException, IndexOutOfBoundsException or ArithmeticException. Each of these are caused when you try to do something that is not possible. For example, a NullPointerException will be thrown when you try to dereference a Null-object:
Object a = null;
a.toString(); //this line throws a NullPointerException
Object[] b = new Object[5];
System.out.println(b[10]); //this line throws an IndexOutOfBoundsException,
//because b is only 5 elements long
int ia = 5;
int ib = 0;
ia = ia/ib; //this line throws an ArithmeticException with the
//message "/ by 0", because you are trying to
//divide by 0, which is not possible.
How should I deal with Stacktraces/Exceptions?
At first, find out what is causing the Exception. Try googling the name of the exception to find out what the cause of that exception is. Most of the time it will be caused by incorrect code. In the given examples above, all of the exceptions are caused by incorrect code. So for the NullPointerException example you could make sure that a is never null at that time. You could, for example, initialise a or include a check like this one:
if (a!=null) {
a.toString();
}
This way, the offending line is not executed if a==null. Same goes for the other examples.
Sometimes you can't make sure that you don't get an exception. For example, if you are using a network connection in your program, you cannot stop the computer from loosing it's internet connection (e.g. you can't stop the user from disconnecting the computer's network connection). In this case the network library will probably throw an exception. Now you should catch the exception and handle it. This means, in the example with the network connection, you should try to reopen the connection or notify the user or something like that. Also, whenever you use catch, always catch only the exception you want to catch, do not use broad catch statements like catch (Exception e) that would catch all exceptions. This is very important, because otherwise you might accidentally catch the wrong exception and react in the wrong way.
try {
Socket x = new Socket("1.1.1.1", 6789);
x.getInputStream().read()
} catch (IOException e) {
System.err.println("Connection could not be established, please try again later!")
}
Why should I not use catch (Exception e)?
Let's use a small example to show why you should not just catch all exceptions:
int mult(Integer a,Integer b) {
try {
int result = a/b
return result;
} catch (Exception e) {
System.err.println("Error: Division by zero!");
return 0;
}
}
What this code is trying to do is to catch the ArithmeticException caused by a possible division by 0. But it also catches a possible NullPointerException that is thrown if a or b are null. This means, you might get a NullPointerException but you'll treat it as an ArithmeticException and probably do the wrong thing. In the best case you still miss that there was a NullPointerException. Stuff like that makes debugging much harder, so don't do that.
TLDR
Figure out what is the cause of the exception and fix it, so that it doesn't throw the exception at all.
If 1. is not possible, catch the specific exception and handle it.
Never just add a try/catch and then just ignore the exception! Don't do that!
Never use catch (Exception e), always catch specific Exceptions. That will save you a lot of headaches.
To add on to what Rob has mentioned. Setting break points in your application allows for the step-by-step processing of the stack. This enables the developer to use the debugger to see at what exact point the method is doing something that was unanticipated.
Since Rob has used the NullPointerException (NPE) to illustrate something common, we can help to remove this issue in the following manner:
if we have a method that takes parameters such as: void (String firstName)
In our code we would want to evaluate that firstName contains a value, we would do this like so: if(firstName == null || firstName.equals("")) return;
The above prevents us from using firstName as an unsafe parameter. Therefore by doing null checks before processing we can help to ensure that our code will run properly. To expand on an example that utilizes an object with methods we can look here:
if(dog == null || dog.firstName == null) return;
The above is the proper order to check for nulls, we start with the base object, dog in this case, and then begin walking down the tree of possibilities to make sure everything is valid before processing. If the order were reversed a NPE could potentially be thrown and our program would crash.
To understand the name: A stack trace is a a list of Exceptions( or you can say a list of "Cause by"), from the most surface Exception(e.g. Service Layer Exception) to the deepest one (e.g. Database Exception). Just like the reason we call it 'stack' is because stack is First in Last out (FILO), the deepest exception was happened in the very beginning, then a chain of exception was generated a series of consequences, the surface Exception was the last one happened in time, but we see it in the first place.
Key 1:A tricky and important thing here need to be understand is : the deepest cause may not be the "root cause", because if you write some "bad code", it may cause some exception underneath which is deeper than its layer. For example, a bad sql query may cause SQLServerException connection reset in the bottem instead of syndax error, which may just in the middle of the stack.
-> Locate the root cause in the middle is your job.
Key 2:Another tricky but important thing is inside each "Cause by" block, the first line was the deepest layer and happen first place for this block. For instance,
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
at com.example.myproject.Author.getBookTitles(Author.java:25)
at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
Book.java:16 was called by Auther.java:25 which was called by Bootstrap.java:14, Book.java:16 was the root cause.
Here attach a diagram sort the trace stack in chronological order.
There is one more stacktrace feature offered by Throwable family - the possibility to manipulate stack trace information.
Standard behavior:
package test.stack.trace;
public class SomeClass {
public void methodA() {
methodB();
}
public void methodB() {
methodC();
}
public void methodC() {
throw new RuntimeException();
}
public static void main(String[] args) {
new SomeClass().methodA();
}
}
Stack trace:
Exception in thread "main" java.lang.RuntimeException
at test.stack.trace.SomeClass.methodC(SomeClass.java:18)
at test.stack.trace.SomeClass.methodB(SomeClass.java:13)
at test.stack.trace.SomeClass.methodA(SomeClass.java:9)
at test.stack.trace.SomeClass.main(SomeClass.java:27)
Manipulated stack trace:
package test.stack.trace;
public class SomeClass {
...
public void methodC() {
RuntimeException e = new RuntimeException();
e.setStackTrace(new StackTraceElement[]{
new StackTraceElement("OtherClass", "methodX", "String.java", 99),
new StackTraceElement("OtherClass", "methodY", "String.java", 55)
});
throw e;
}
public static void main(String[] args) {
new SomeClass().methodA();
}
}
Stack trace:
Exception in thread "main" java.lang.RuntimeException
at OtherClass.methodX(String.java:99)
at OtherClass.methodY(String.java:55)
Just to add to the other examples, there are inner(nested) classes that appear with the $ sign. For example:
public class Test {
private static void privateMethod() {
throw new RuntimeException();
}
public static void main(String[] args) throws Exception {
Runnable runnable = new Runnable() {
#Override public void run() {
privateMethod();
}
};
runnable.run();
}
}
Will result in this stack trace:
Exception in thread "main" java.lang.RuntimeException
at Test.privateMethod(Test.java:4)
at Test.access$000(Test.java:1)
at Test$1.run(Test.java:10)
at Test.main(Test.java:13)
The other posts describe what a stack trace is, but it can still be hard to work with.
If you get a stack trace and want to trace the cause of the exception, a good start point in understanding it is to use the Java Stack Trace Console in Eclipse. If you use another IDE there may be a similar feature, but this answer is about Eclipse.
First, ensure that you have all of your Java sources accessible in an Eclipse project.
Then in the Java perspective, click on the Console tab (usually at the bottom). If the Console view is not visible, go to the menu option Window -> Show View and select Console.
Then in the console window, click on the following button (on the right)
and then select Java Stack Trace Console from the drop-down list.
Paste your stack trace into the console. It will then provide a list of links into your source code and any other source code available.
For example, if we had this program:
public class ExceptionTest {
public static void main(String[] args) {
int l = trimmedLength(null);
System.out.println("Trimmed length = " + l);
}
private static int trimmedLength(String string) {
return string.trim().length();
}
}
You would get this stack trace:
The most recent method call made (and the one that caused the exception) will be the top of the stack, which is the top line (excluding the error message text). In this case, that is the trimmedLength method. Going down the stack goes back in time. The second line is the method that calls the first line, etc.
If you are using open-source software, you might need to download and attach to your project the sources if you want to examine. Download the source jars, in your project, open the Referenced Libraries folder to find your jar for your open-source module (the one with the class files) then right click, select Properties and attach the source jar.
I need check currentFile of MIME-type. If result is success and file have MIME-type return true. If wasn't checking succed return false.
With this goal I use JMimeMagic.
I try do this according this post
Output from this code is - net.sf.jmimemagic.MagicMatchNotFoundException
You need have JDK 7 - for changing File to byte[] at this way(Files.readAllBytes(path)).
Code:
class ProbeContentTypeCheker implements Checker {
#Override
public boolean check(File currentFile) {
String mimeType = null;
try {
Path path = Paths.get(currentFile.getAbsolutePath());
byte[] data = Files.readAllBytes(path);
MagicMatch match = Magic.getMagicMatch(data);
mimeType = match.getMimeType();
} catch (MagicParseException | MagicMatchNotFoundException
| MagicException | IOException e) {
e.printStackTrace();
}
if (null != mimeType) {
return true;
}
return false;
}
}
Output (only if it's "wrong" type):
net.sf.jmimemagic.MagicMatchNotFoundException
at net.sf.jmimemagic.Magic.getMagicMatch(Magic.java:222)
at net.sf.jmimemagic.Magic.getMagicMatch(Magic.java:170)
at task.ProbeContentTypeCheker.check(FileScan.java:357)
at task.FolderScan.findFiles(FileScan.java:223)
at task.FolderScan.findFiles(FileScan.java:215)
at task.FolderScan.run(FileScan.java:202)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
If file is "ok" type => output to console normal. But after some time arise another exception:
Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.toCharArray(String.java:2753)
at org.apache.oro.text.perl.Perl5Util.match(Unknown Source)
at net.sf.jmimemagic.MagicMatcher.testRegex(MagicMatcher.java:663)
at net.sf.jmimemagic.MagicMatcher.testInternal(MagicMatcher.java:433)
at net.sf.jmimemagic.MagicMatcher.test(MagicMatcher.java:341)
at net.sf.jmimemagic.Magic.getMagicMatch(Magic.java:208)
at net.sf.jmimemagic.Magic.getMagicMatch(Magic.java:170)
at task.ProbeContentTypeCheking.check(FileScan.java:384)
at task.FolderScan.findFiles(FileScan.java:228)
at task.FolderScan.findFiles(FileScan.java:225)
at task.FolderScan.findFiles(FileScan.java:225)
at task.FolderScan.run(FileScan.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Question:
How do solve this arises of exception?
JMimeMagic 0.1.2 depends on Commons Logging 1.0.4
A NoClassDefFoundError means that the Java Virtual Machine or a ClassLoader instance tries to load in the definition of a class (as part of a normal method call or as part of creating a new instance using the new expression) and no definition of the class could be found.
The solution would be to add the commons-logging-1.0.4.jar to your classpath.
Note that JMimeMagic has other 3rd party dependencies:
Jakarta ORO 2.0.8
Log4j 1.2.8
Xerces 2.4.0 (optional)
xml-apis 2.0.2
xmlParserAPIs 2.0.2
Update - MagicMatchNotFoundException
The MagicMatchNotFoundException is thrown if no mime type match is found for the provided data.
You can set the log level of net.sf.jmimemagic to DEBUG to get more information about what is going on
Update 2 - OutOfMemoryError
The OOM looks related to the behavior of JmimeMagic. In some cases it will try to run a regular expression against the entire byte array input to find the magic number match. See this reported issue for the Nuxeo Enterprise Platform.
I think you can solve this issue by limiting the size of the byte array you pass to getMagicMatch
I switched an existing code base to Java 7 and I keep getting this warning:
warning: File for type '[Insert class here]' created in the last round
will not be subject to annotation processing.
A quick search reveals that no one has hit this warning.
It's not documented in the javac compiler source either:
From OpenJDK\langtools\src\share\classes\com\sun\tools\javac\processing\JavacFiler.java
private JavaFileObject createSourceOrClassFile(boolean isSourceFile, String name) throws IOException {
checkNameAndExistence(name, isSourceFile);
Location loc = (isSourceFile ? SOURCE_OUTPUT : CLASS_OUTPUT);
JavaFileObject.Kind kind = (isSourceFile ?
JavaFileObject.Kind.SOURCE :
JavaFileObject.Kind.CLASS);
JavaFileObject fileObject =
fileManager.getJavaFileForOutput(loc, name, kind, null);
checkFileReopening(fileObject, true);
if (lastRound) // <-------------------------------TRIGGERS WARNING
log.warning("proc.file.create.last.round", name);
if (isSourceFile)
aggregateGeneratedSourceNames.add(name);
else
aggregateGeneratedClassNames.add(name);
openTypeNames.add(name);
return new FilerOutputJavaFileObject(name, fileObject);
}
What does this mean and what steps can I take to clear this warning?
Thanks.
The warning
warning: File for type '[Insert class here]' created in the last round
will not be subject to annotation processing
means that your were running an annotation processor creating a new class or source file using a javax.annotation.processing.Filer implementation (provided through the javax.annotation.processing.ProcessingEnvironment) although the processing tool already decided its "in the last round".
This may be problem (and thus the warning) because the generated file itself may contain annotations being ignored by the annotation processor (because it is not going to do a further round).
The above ought to answer the first part of your question
What does this mean and what steps can I take to clear this warning?
(you figured this out already by yourself, didn't you :-))
What possible steps to take? Check your annotation processors:
1) Do you really have to use filer.createClassFile / filer.createSourceFile on the very last round of the annotaion processor? Usually one uses the filer object inside of a code block like
for (TypeElement annotation : annotations) {
...
}
(in method process). This ensures that the annotation processor will not be in its last round (the last round always being the one having an empty set of annotations).
2) If you really can't avoid writing your generated files in the last round and these files are source files, trick the annotation processor and use the method "createResource" of the filer object (take "SOURCE_OUTPUT" as location).
In OpenJDK test case this warning produced because processor uses "processingOver()" to write new file exactly at last round.
public boolean process(Set<? extends TypeElement> elems, RoundEnvironment renv) {
if (renv.processingOver()) { // Write only at last round
Filer filer = processingEnv.getFiler();
Messager messager = processingEnv.getMessager();
try {
JavaFileObject fo = filer.createSourceFile("Gen");
Writer out = fo.openWriter();
out.write("class Gen { }");
out.close();
messager.printMessage(Diagnostic.Kind.NOTE, "File 'Gen' created");
} catch (IOException e) {
messager.printMessage(Diagnostic.Kind.ERROR, e.toString());
}
}
return false;
}
I modified original example code a bit. Added diagnostic note "File 'Gen' created", replaced "*" mask with "org.junit.runner.RunWith" and set return value to "true". Produced compiler log was:
Round 1:
input files: {ProcFileCreateLastRound}
annotations: [org.junit.runner.RunWith]
last round: false
Processor AnnoProc matches [org.junit.runner.RunWith] and returns true.
Round 2:
input files: {}
annotations: []
last round: true
Note: File 'Gen' created
Compilation completed successfully with 1 warning
0 errors
1 warning
Warning: File for type 'Gen' created in the last round will not be subject to annotation processing.
If we remove my custom note from log, it's hard to tell that file 'Gen' was actually created on 'Round 2' - last round. So, basic advice applies: if in doubt - add more logs.
Where is also a little bit of useful info on this page:
http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/javac.html
Read section about "ANNOTATION PROCESSING" and try to get more info with compiler options:
-XprintProcessorInfo
Print information about which annotations a processor is asked to process.
-XprintRounds Print information about initial and subsequent annotation processing rounds.
I poked around the java 7 compiler options and I found this:
-implicit:{class,none}
Controls the generation of class files for implicitly loaded source files. To automatically generate class files, use -implicit:class. To suppress class file generation, use -implicit:none. If this option is not specified, the default is to automatically generate class files. In this case, the compiler will issue a warning if any such class files are generated when also doing annotation processing. The warning will not be issued if this option is set explicitly. See Searching For Types.
Source
Can you try and implicitly declare the class file.
I'm trying to validate an Atom feed with Java 5 (JRE 1.5.0 update 11). The code I have works without problem in Java 6, but fails when running in Java 5 with a
org.xml.sax.SAXParseException: src-resolve: Cannot resolve the name 'xml:base' to a(n) 'attribute declaration' component.
I think I remember reading something about the version of Xerces bundled with Java 5 having some problems with some schemas, but i cant find the workaround. Is it a known problem ? Do I have some error in my code ?
public static void validate() throws SAXException, IOException {
List<Source> schemas = new ArrayList<Source>();
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/dc.xsd")));
// Lookup a factory for the W3C XML Schema language
SchemaFactory factory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
// Compile the schemas.
Schema schema = factory.newSchema(schemas.toArray(new Source[schemas.size()]));
Validator validator = schema.newValidator();
// load the file to validate
Source source = new StreamSource(AtomValidator.class.getResourceAsStream("/sample-feed.xml"));
// check the document
validator.validate(source);
}
Update : I tried the method below, but I still have the same problem if I use Xerces 2.9.0. I also tried adding xml.xsd to the list of schemas (as xml:base is defined in xml.xsd) but this time I have
Exception in thread "main" org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document 'null', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Update 2: I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works. I'll try to post a "real answer" from home.
and sorry, I cant reply as a comment : because of security reasons XHR is disabled from work ...
Indeed, people have been mentioning the Java 5 Sun provided SchemaFactory is giving troubles.
So: did you include Xerces in your project yourself?
After including Xerces, you need to ensure it is being used. If you like to hardcode it (well, as a minimal requirement you'd probably use some application properties file to enable and populate the following code):
String schemaFactoryProperty =
"javax.xml.validation.SchemaFactory:" + XMLConstants.W3C_XML_SCHEMA_NS_URI;
System.setProperty(schemaFactoryProperty,
"org.apache.xerces.jaxp.validation.XMLSchemaFactory");
SchemaFactory factory =
SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Or, if you don't want to hardcode, or when your troublesome code would be in some 3rd party library that you cannot change, set it on the java command line or environment options. For example (on one line of course):
set JAVA_OPTS =
"-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema
=org.apache.xerces.jaxp.validation.XMLSchemaFactory"
By the way: apart from the Sun included SchemaFactory implementation giving trouble (something like com.sun.org.apache.xerces.internal.jaxp.validation.xs.schemaFactoryImpl), it also seems that the "discovery" of non-JDK implementations fails in that version. If I understand correctly than, normally, just including Xerces would in fact make SchemaFactory#newInstance find that included library, and give it precedence over the Sun implementation. To my knowledge, that fails as well in Java 5, making the above configuration required.
I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works.
Ah, I didn't realize that xml.xsd is in fact the one referenced as http://www.w3.org/2001/xml.xsd or something like that. That should teach us to always show some XML and XSD fragments as well. ;-)
So, am I correct to assume that 1.) to fix the Java 5 issue, you still needed to include Xerces and set the system property, and that 2.) you did not have xml.xsd available locally?
Before you found your solution, did you happen to try using getResource rather than getResourceAsStream, to see if the exception would then have showed you some more details?
If you actually did have xml.xsd available (so: if getResource did in fact yield a URL) then I wonder what Xerces was trying to fetch from the internet then. Or maybe you did not add that schema to the list prior to adding your own schemas? The order is important: dependencies must be added first.
For whoever gets tot his question using the search: maybe using a custom EntityResolver could have indicated the source of the problem as well (if only writing something to the log and just returning null to tell Xerces to use the default behavior).
Hmmm, just read your "comment" -- editing does not alert people for new replies, so time to ask your boss for some iPhone or some other gadget that is connected to the net directly ;-)
Well, I assume you added:
schemas.add(
new StreamSource(AtomValidator.class.getResourceAsStream("/xml.xsd")));
If so, is xml.xsd actually to be found on the classpath then? I wonder if the getResourceAsStream did not yield null in your case, and how new StreamSource(null) would act then.
Even if getResourceAsStream did not yield null, the resulting StreamSource would still not know where it was loaded from, which may be a problem when trying to include references. So, what if you use the constructor StreamSource(String systemId) instead:
schemas.add(new StreamSource(AtomValidator.class.getResource("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResource("/dc.xsd")));
You might also use StreamSource(InputStream inputStream, String systemId), but I don't see any advantage over the above two lines. However, the documentation explains why passing the systemId in either of the 2 constructors seems good:
This constructor allows the systemID to be set in addition to the input stream, which allows relative URIs to be processed.
Likewise, setSystemId(String systemId) explains a bit:
The system identifier is optional if there is a byte stream or a character stream, but it is still useful to provide one, since the application can use it to resolve relative URIs and can include it in error messages and warnings (the parser will attempt to open a connection to the URI only if there is no byte stream or character stream specified).
If this doesn't work out, then maybe some custom error handler can give you more details:
ErrorHandlerImpl errorHandler = new ErrorHandlerImpl();
validator.setErrorHandler(errorHandler);
:
:
validator.validate(source);
if(errorHandler.hasErrors()){
LOG.error(errorHandler.getMessages());
throw new [..];
}
if(errorHandler.hasWarnings()){
LOG.warn(errorHandler.getMessages());
}
...using the following ErrorHandler to capture the validation errors and continue parsing as far as possible:
import org.xml.sax.helpers.DefaultHandler;
private class ErrorHandlerImpl extends DefaultHandler{
private String messages = "";
private boolean validationError = false;
private boolean validationWarning = false;
public void error(SAXParseException exception) throws SAXException{
messages += "Error: " + exception.getMessage() + "\n";
validationError = true;
}
public void fatalError(SAXParseException exception) throws SAXException{
messages += "Fatal: " + exception.getMessage();
validationError = true;
}
public void warning(SAXParseException exception) throws SAXException{
messages += "Warn: " + exception.getMessage();
validationWarning = true;
}
public boolean hasErrors(){
return validationError;
}
public boolean hasWarnings(){
return validationWarning;
}
public String getMessages(){
return messages;
}
}