I have a very strange problem with NullPointerException. Code example is as follows:
...
... public String[] getParams(...) {
... ...
... ...
143 return new String[] {
144 getUserFullName(),
145 StringUtil.formatDate(sent),
. tiltu,
. StringUtil.initCap(user.getName()),
. vuosi.toString(),
. asiatyyppi[0] + " " + lisatiedot[0],
. asiatyyppi[1] + " " + lisatiedot[1],
. alaviitteet[0],
152 alaviitteet[1]};
153 }
Now, I have got an issue from production having a stack trace:
java.lang.NullPointerException
at package.EmailService.getParams(EmailService.java:143)
...
I am unable to produce that kind of stack trace myself. It maybe some environment issue that for some reason line numbers don't match. If I have null references on any variable stack trace points to that specific line but never to line 143.
But what I want to ask is: is it possible to produce NullPointerException at line 143 specifically?
The line number in the stack trace comes from the LineNumberTable attribute in the class file. (See JVM specification)
It would be no problem to output the right line number for a subexpression - all the compiler has to do is to say that from byte-code index x to y, there is a correspondence with source code line z.
But up to and including Java 1.7 there was a bug in the compiler, that was fixed in 1.8:
https://bugs.openjdk.java.net/browse/JDK-7024096
A DESCRIPTION OF THE PROBLEM : linenumbertable in compiled code only
has line numbers for starts of statements. When a statement is using
method chaining or other "fluent Interface" style API's a single
statement can span tens of lines and contain hundreds on method
invocations.
Currently an exception throw from within any of these methods will
have the linenumber of the first line of the enclosing statement which
makes it very hard to debug which method call is having a problem.
linnumbertable should have correct line numbers for every method
invocation.
--
BT2:EVALUATION
It seems simple enough to fix this, but its kinda risky at the end of
the jdk7 development cycle, targetting to jdk8.
So in 1.7, you would get the wrong reported line number for these kind of subexpressions (if they occurred within the same method though - if you invoked another method in a subexpression, and that other method caused a NullPointerException, you would see it reported there - this is probably why the bug isn't always a big problem)
One way you could work around this is by taking the Java 8 compiler to compile your source code, and use the flags javac -source 1.7 -target 1.7. But it would be better and safer to upgrade your prod environment to 1.8.
Consider your original code which defines a new String array:
return new String[] {
getUserFullName(),
StringUtil.formatDate(sent),
tiltu,
StringUtil.initCap(user.getName()),
vuosi.toString(),
asiatyyppi[0] + " " + lisatiedot[0],
asiatyyppi[1] + " " + lisatiedot[1],
alaviitteet[0],
alaviitteet[1]};
}
If any of the elements of the inline array should trigger a NullPointerException the JVM will interpret the Exception as having occurred on the line where the definition began. In other words, the JVM will view the above code as being the same as:
return new String[] { getUserFullName(), StringUtil.formatDate(sent), tiltu, StringUtil.initCap user.getName()), vuosi.toString(), asiatyyppi[0] + " " + lisatiedot[0], asiatyyppi[1] + " " + lisatiedot[1], alaviitteet[0], alaviitteet[1]}; }
where everything is on one line.
If you really want to handle NullPointerExceptions here, you should define the variables outside the instantiaition.
Related
What am I doing?
I am writing a data analysis program in Java which relies on R´s arulesViz library to mine association rules.
What do I want?
My purpose is to store the rules in a String variable in Java so that I can process them later.
How does it work?
The code works using a combination of String.format and eval Java and RJava instructions respectively, being its behavior summarized as:
Given properly formatted Java data structures, creates a data frame in R.
Formats the recently created data frame into a transaction list using the arules library.
Runs the apriori algorithm with the transaction list and some necessary values passed as parameter.
Reorders the generated association rules.
Given that the association rules cannot be printed, they are written to the standard output with R´s write method, capture the output and store it in a variable. We have converted the association rules into a string variable.
We return the string.
The code is the following:
// Step 1
Rutils.rengine.eval("dataFrame <- data.frame(as.factor(c(\"Red\", \"Blue\", \"Yellow\", \"Blue\", \"Yellow\")), as.factor(c(\"Big\", \"Small\", \"Small\", \"Big\", \"Tiny\")), as.factor(c(\"Heavy\", \"Light\", \"Light\", \"Heavy\", \"Heavy\")))");
//Step 2
Rutils.rengine.eval("transList <- as(dataFrame, 'transactions')");
//Step 3
Rutils.rengine.eval(String.format("info <- apriori(transList, parameter = list(supp = %f, conf = %f, maxlen = 2))", supportThreshold, confidenceThreshold));
// Step 4
Rutils.rengine.eval("orderedRules <- sort(info, by = c('count', 'lift'), order = FALSE)");
// Step 5
REXP res = Rutils.rengine.eval("rulesAsString <- paste(capture.output(write(orderedRules, file = stdout(), sep = ',', quote = TRUE, row.names = FALSE, col.names = FALSE)), collapse='\n')");
// Step 6
return res.asString().replaceAll("'", "");
What´s wrong?
Running the code in Linux Will work perfectly, but when I try to run it in Windows, I get the following error referring to the return line:
Exception in thread "main" java.lang.NullPointerException
This is a common error I have whenever the R code generates a null result and passes it to Java. There´s no way to syntax check the R code inside Java, so whenever it´s wrong, this error message appears.
However, when I run the R code in brackets in the R command line in Windows, it works flawlessly, so both the syntax and the data flow are OK.
Technical information
In Linux, I am using R with OpenJDK 10.
In Windows, I am currently using Oracle´s latest JDK release, but trying to run the program with OpenJDK 12 for Windows does not solve anything.
Everything is 64 bits.
The IDE used in both operating systems is IntelliJ IDEA 2019.
Screenshots
Linux run configuration:
Windows run configuration:
When I call eval (in strict mode) on a nashorn engine with the following script I get an exception:
var yfunc = function () {
(null).apply(null, arguments);
};
yfunc();
I've truncated my personal situation heavily. The "(null)" on line 2 can be replaced with anything between parenthesis or a local variable, either way just something that shouldn't throw a compile error, and it will yield the same result.
The issue seems to be explicitly that "arguments" is passed directly as the second argument of calling a method called "apply". Any of the following changes will undo the thrown exception:
Putting "arguments" in a variable first (but simply wrapping it in parenthesis doesn't work!)
Calling something other than apply
Passing "arguments" in a different argument slot when calling apply
Calling print() (with or without passing any arguments) as a preceding line of code inside yfunc() (weird huh?)
Defining more than 0 parameters for yfunc()
Binding yfunc first and then calling the bound method
Calling yfunc via Function.apply (not so much with Function.call!)
The Exception thrown is this:
Exception in thread "main" java.lang.ClassCastException: Cannot cast jdk.nashorn.internal.runtime.Undefined to jdk.nashorn.internal.runtime.ScriptFunction
at java.lang.invoke.MethodHandleImpl.newClassCastException(MethodHandleImpl.java:361)
at java.lang.invoke.MethodHandleImpl.castReference(MethodHandleImpl.java:356)
at jdk.nashorn.internal.scripts.Script$\^eval\_.:program(<eval>:4)
at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
When I call this method with an owner, the exception thrown changes. Example code:
var yfunc = {
method: function () {
(null).apply(null, arguments);
}
};
var x = yfunc.method();
Then the thrown exception looks like this:
Exception in thread "main" java.lang.ClassCastException: Cannot cast jdk.nashorn.internal.scripts.JO4 to jdk.nashorn.internal.runtime.ScriptFunction
at java.lang.invoke.MethodHandleImpl.newClassCastException(MethodHandleImpl.java:361)
at java.lang.invoke.MethodHandleImpl.castReference(MethodHandleImpl.java:356)
at jdk.nashorn.internal.scripts.Script$\^eval\_.:program(<eval>:5)
at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
I've reproduced the issue so far on specifically these environments:
windows 7 64bit -> jdk 1.8.0_60 64bit
windows 8 64bit -> jdk 1.8.0_131 64bit
I can't seem to find anything on the internet about similar issues. Do I need to report this to Oracle/OpenJDK?
Minor update
Added items 6 and 7 to list of "following changes will undo the thrown exception".
Final update
Bug filed: JDK-8184720
Yes, it appears to be a bug. Please file a bug.
I'm trying to track down the source of an EOFException in some client/server code, but many of the classes used only have methods which say that they throw an IOException, not specifically an EOFException. Normally I'd just look at the stack trace but I don't have it and can't reproduce it. So, it would be helpful to have a list of every class which has methods which throw EOFException specifically, but I don't know how to find out this information.
Is there a source of information on every (standard) Java class (in SE 7, in my case) which has methods which throw EOFException? I have tried reading the Javadocs on EOFException to no avail.
(I have found, at least, that DataInput does mentionEOFException. But are there any more?)
Afterword: Since it seems the only way to find out is to inspect the source code, here is the result of me searching for "EOFException" in the extracted source code (1.7.0_45).
EDIT: Added results for 7.51, for all source code in the JDK, at the bottom.
In Java 6.17, here are all classes that explicitly throw java.io.EOFException. Specifically, it is the number of times new\s+EOFException was found in each source-code file.
java.io.DataInputStream.java: 8
java.io.ObjectInputStream.java: 6
java.io.RandomAccessFile.java: 8
java.util.zip.GZIPInputStream.java: 2
java.util.zip.InflaterInputStream.java: 1
java.util.zip.ZipFile.java: 1
java.util.zip.ZipInputStream.java: 1
Here is another bit of information: All source-code files containing catch\s*\(\s*EOFException:
java.io.ObjectInputStream.java: 1
java.util.zip.ZipInputStream.java: 1
Note that there are no standard java.* Exception classes that extend EOFException (there are no occurances of
extends\s+([a-z]+\.)*EOFException
anywhere in the java.* source-code).
This is a limited but valuable starting point. As mentioned by others, there may be situations that this misses--when all you have is the source-code to work from, it will be time-consuming to find them. Hopefully this information will set you down the right path.
Here are the results for Java 7.51, for all source-code provided with the JDK:
extends\s+([a-z]+\.)*EOFException
none
catch\s*\(\s*EOFException
com.sun.imageio.plugins.gif.GIFImageReader: 1
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl: 1
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl: 4
com.sun.org.apache.xerces.internal.impl.XMLVersionDetector: 1
com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDLoader: 2
java.io.ObjectInputStream: 1
java.util.zip.ZipInputStream: 1
new\s+EOFException
com.sun.corba.se.impl.io.IIOPInputStream: 1
com.sun.imageio.plugins.png.PNGImageReader: 1
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl: 1
com.sun.org.apache.xerces.internal.impl.XMLEntityManager: 1
com.sun.org.apache.xerces.internal.impl.XMLEntityScanner: 1
java.io.DataInputStream: 8
java.io.ObjectInputStream: 6
java.io.RandomAccessFile: 8
java.util.zip.GZIPInputStream: 2
java.util.zip.InflaterInputStream: 1
java.util.zip.ZipFile: 1
java.util.zip.ZipInputStream: 1
javax.imageio.stream.ImageInputStreamImpl: 8
You can find that out only under the assumption that the code creates and immediately throws an EOFException. But the follwoing could also throw an EOFException:
class Foo {
...
public void iAmHarmless(Exception x) { if (x != null) throw x; }
...
}
Or how about the follwoing, to defeat aliteralminds method:
class XYException extends EOFException { ... }
class Foo {
public void surprise() { throw new XYException().super(); }
}
Joking aside - methods from many classes may throw EOFException simply because of inheritance. A grep for "EOFException" gives only the very base classes. But you must consider all subclasses thereof. Example: java.util.jar.JarInputStream
I am trying to generate a method named hello that returns the value 2 using dynamic bytecode generation. This is my current code. To generate the method.
dout.writeShort(Modifier.PUBLIC);//class modifier
dout.writeShort(classConstant("test"));//class name
dout.writeShort(classConstant(Object.class.getName()));//superclass
dout.writeShort(0);//interface count
dout.writeShort(0);//field count
dout.writeShort(1);//method count
dout.writeShort(Modifier.PUBLIC|Modifier.STATIC);//modifiers
dout.writeShort(utfConstant("test"));//name
dout.writeShort(utfConstant(methodDescriptor(int.class, new Class[]{})));//descriptor
dout.writeShort(1);//attribute count
dout.writeShort(utfConstant("Code"));//attribute name
dout.writeInt(34);//attribute length
dout.writeShort(1);//max stack
dout.writeShort(0);//max locals
dout.writeInt(2);//code length
dout.writeByte(0x05);//iconst_2 opcode
dout.writeByte(0xAC);//ireturn opcode
dout.writeShort(0);//exception count
dout.writeShort(0);//attribute count
dout.writeShort(0);//class attributes
The problem is that when i run this code, i get this exception
Exception in thread "main" java.lang.ClassFormatError: Invalid method Code length 0 in class file test
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.lang.ClassLoader.defineClass(ClassLoader.java:634)
at Bytecode.BytecodeTest$BytecodeClassLoader.buildClass(BytecodeTest.java:229)
at Bytecode.BytecodeTest.makeClass(BytecodeTest.java:42)
at Bytecode.BytecodeTest.buildClass(BytecodeTest.java:27)
at Bytecode.BytecodeTest.main(BytecodeTest.java:19)
The weird thing is i am making the code length greater than 0 i am making it 2. I went back through the oracle specification but it still looks right. I have a feeling that i am writing some of the data as the wrong type, but i still cant find a problem.
An undocumented feature of the Hotspot verifier is that for versions <= 45.2, it uses shorter field lengths for some of the fields in the code attribute. That's why changing the version to 49 fixed everything.
If you use Krakatau, it will automatically take care of this, but I haven't seen any other tools that handle this case.
Luckily, the first stable public version of Java was 45.3, so you are unlikely to see legitimate code like this in the wild. But it is a neat trick for foiling reverse engineers.
Well, one thing that strikes me is the attribute length: by my count, it should be 14 (not 34). You also seem to be missing the class attribute count.
It would probably help you to define a couple helper methods for writing attributes, to ensure that you are computing and writing the length correctly, e.g., something like this:
private int writeAttribute(final String attributeName) {
dout.putShort(utfConstant(attributeName));
dout.putInt(0);
return dout.position();
}
private void endAttribute(final int attributeStart) {
dout.putInt(attributeStart- 4, dout.position() - attributeStart);
}
private void writeCode() {
final int codeAttributeStart = writeAttribute("Code");
dout.writeShort(1);//max stack
dout.writeShort(0);//max locals
dout.writeInt(2);//code length
dout.writeByte(0x05);//iconst_2 opcode
dout.writeByte(0xAC);//ireturn opcode
dout.writeShort(0);//exception count
dout.writeShort(0);//attribute count
endAttribute(codeAttributeStart);
}
Also, make sure the classfile minor/major version you're writing out matches the specification you're following--the format does change from time to time :).
I am using scanner class in java5, and the following code will throw an exception:
Scanner scanner = new Scanner
(new File(args[0]));
int dealId;
while (scanner.hasNextLine()) {
dealId = scanner.nextInt();
System.out.println(dealId);
}
scanner.close();
The stacktrace is:
Exception in thread "main" java.lang.NullPointerException
at java.util.regex.Matcher.toMatchResult(libgcj.so.10)
at java.util.Scanner.myCoreNext(libgcj.so.10)
at java.util.Scanner.myPrepareForNext(libgcj.so.10)
at java.util.Scanner.myNextLine(libgcj.so.10)
at java.util.Scanner.hasNextLine(libgcj.so.10)
Does anybody knows what caused this exception?
The GCJ Home Page suggest it "supports most of the 1.4 libraries plus some 1.5 additions. "
Scanner was added in version 1.5 and I suspect you have hit a piece of functionality GCJ doesn't support. You need to try something different to see what you can get to work.
Is there any reason you are not using OpenJDK/Oracle Java 6 or 7? (Please don't say its for performance reasons ;)
I reproduced the error and found a work around
Here is the code, compiled on x86_64 GNU/Linux, Fedora with Java 1.5.0:
Scanner r = new Scanner(f, "ISO-8859-1");
while(r.hasNext()){
String line = r.nextLine(); //The guts of nextLine(), specifically:
//Matcher.toMatchResult bubbles up a
//nullPointerException
}
The file just contains two ascii words separated by a newline. The runtime Exception only occurs when nextLine processes the last line of the file, whether it has characters or not:
java.lang.NullPointerException
at java.util.regex.Matcher.toMatchResult(libgcj.so.10)
at java.util.Scanner.myCoreNext(libgcj.so.10)
at java.util.Scanner.myPrepareForNext(libgcj.so.10)
at java.util.Scanner.myNextLine(libgcj.so.10)
at java.util.Scanner.nextLine(libgcj.so.10)
at Main.parseFile(Main.java:1449)
at Main.construct(Main.java:1420)
at Main.populateBlogPosts(Main.java:1399)
at Main.main(Main.java:263)
Here is a bug report on this issue: https://bugs.openjdk.java.net/browse/JDK-6178785
Diagnosis
It's a bug in libgcj.so.10, perfectly legitimate ascii input as well as blankstring causes it to puke an NPE on the last line of a file.
Workaround
Since this bug only occurs on the very last line of the file, the hacky workaround is to initially make sure there is at least one newline at the end of the file, then catch and ignore the nullPointerException bubbled up from toMatchResult and exit the loop when that happens.