Develop Reference

NoSuchMethodError in neo4j when calling the DatabaseManagementServiceBuilder( path ).build() method

I´m devolopping a Java application with neo4j embedded. I don´t know why but it is not possible to build a Database. When I call the the build()-Method in the DatabaseManagementServiceBuilder class I get the following warnings and exeption:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access using Lookup on org.neo4j.internal.unsafe.UnsafeUtil (file:/C:/Users/Benny_Uni/eclipse-workspace/BSc_cocitation_coauthor_graph_B_Birner_9719989/libs/neo4j-community-4.4.13-windows/neo4j-community-4.4.13/lib/neo4j-unsafe-4.4.13.jar) to class java.nio.Buffer
WARNING: Please consider reporting this to the maintainers of org.neo4j.internal.unsafe.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
at org.neo4j.cypher.internal.options.CypherVersion$.<init>(CypherQueryOptions.scala:131)
at org.neo4j.cypher.internal.options.CypherVersion$.<clinit>(CypherQueryOptions.scala)
at org.neo4j.cypher.internal.config.CypherConfiguration.<init>(CypherConfiguration.scala:60)
at org.neo4j.cypher.internal.config.CypherConfiguration$.fromConfig(CypherConfiguration.scala:40)
at org.neo4j.cypher.internal.config.CypherConfiguration.fromConfig(CypherConfiguration.scala)
at org.neo4j.fabric.bootstrap.FabricServicesBootstrap.bootstrapServices(FabricServicesBootstrap.java:144)
at org.neo4j.graphdb.factory.module.edition.CommunityEditionModule.bootstrapFabricServices(CommunityEditionModule.java:401)
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:148)
at org.neo4j.dbms.api.DatabaseManagementServiceBuilder.newDatabaseManagementService(DatabaseManagementServiceBuilder.java:101)
at org.neo4j.dbms.api.DatabaseManagementServiceBuilder.build(DatabaseManagementServiceBuilder.java:94)
at HeaderEx.getHeader(HeaderEx.java:37)
at PdfParser.main(PdfParser.java:46)
By this code:
String db_path = System.getProperty("user.dir") + File.separator + "author_database";
File database = new File(db_path);
Path paths = Paths.get(db_path);
managementService = new DatabaseManagementServiceBuilder(paths).build();
graphDb = managementService.database( "DEFAULT_DATABASE_NAME" );
I don´t know, what the problem is. It would be very nice if someone can help me.
I tried to clean the project. I changed the neo4j version. I checkt the configurations rgarding the JDK and JVM. I´m using JDK 11. However, nothing works.

Itext Java 11: Illegal reflective access by com.itextpdf.io.source.ByteBufferRandomAccessSource$1

Recently upgraded to Java 11 and began performing a regression check. Currently get an Illegal reflective access error when trying to call com.itextpdf.text.pdf.PdfReader.close. Currently on Itext version 5.5.13 but also tried on itext 7.0.0 and had same issue.
Does anyone have any suggestions on how to fix compatibility issues between Java-11 and Itext?
WARNING: An illegal reflective access operation has occurred WARNING:
Illegal reflective access by
com.itextpdf.io.source.ByteBufferRandomAccessSource$1
(file:...repository/com/itextpdf/io/7.0.0/io-7.0.0.jar) to method
java.nio.DirectByteBuffer.cleaner() WARNING: Please consider reporting
this to the maintainers of
com.itextpdf.io.source.ByteBufferRandomAccessSource$1 WARNING: Use
--illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be
denied in a future release

While I second the comments encouraging you to debug the code and find the root cause (and then submit a pull request), or creating an issue in iText Jira if you are a customer with a support contract (which would raise the priority of the issue), here is a workaround suggestion (that I haven't tested, but I chances are it will work):
Use PdfReader and PdfWriter constructors that accept InputStream and OutputStream, respectively. In this case the code causing the problem should not be invoked. Same thing for all the other cases in which iText interacts with your file system - wrap everything into InputStream/OutputStream, or deal with byte[] arrays.
So this line:
new PdfDocument(new PdfReader(inFilePath), new PdfWriter(outFilePath))
becomes this one:
new PdfDocument(new PdfReader(new FileInputStream(inFilePath)),
new PdfWriter(new FileOutputStream(outFilePath)))
You might also want to wrap the streams into BufferedInputStream/BufferedOutputStream.
Similarly, when dealing with PdfFontFactory, use methods that accept byte[] instead of String representing file path and so on.

If you're curious what is the Illegal Reflective Access is all about, please refer here: what is an illegal reflective access
This particular warning comes from this class:
https://github.com/itext/itext7/blob/develop/io/src/main/java/com/itextpdf/io/source/ByteBufferRandomAccessSource.java
From this particular method:
private static boolean clean(final java.nio.ByteBuffer buffer) {
if (buffer == null || !buffer.isDirect())
return false;
Boolean b = AccessController.doPrivileged(new PrivilegedAction<Boolean>() {
public Boolean run() {
Boolean success = Boolean.FALSE;
try {
// java 9
if (UNMAP_SUPPORTED)
CLEANER.freeBuffer(buffer.toString(), buffer);
// java 8 and lower
else {
Method getCleanerMethod = buffer.getClass().getMethod("cleaner", (Class<?>[]) null);
getCleanerMethod.setAccessible(true);
Object cleaner = getCleanerMethod.invoke(buffer, (Object[]) null);
Method clean = cleaner.getClass().getMethod("clean", (Class<?>[]) null);
clean.invoke(cleaner, (Object[]) null);
}
success = Boolean.TRUE;
} catch (Exception e) {
// This really is a show stopper on windows
Logger logger = LoggerFactory.getLogger(ByteBufferRandomAccessSource.class);
logger.debug(e.getMessage());
}
return success;
}
});
return b;
}
This line exactly:
getCleanerMethod.setAccessible(true);
As long as this warning does not prevent iText from working as expected, I think the best you can do is to submit an issue/PR to iText team and wait for the proper fix to be available.

How to check bytecode length of java method

At this moment I participate in big legacy project with many huge classes and generated code.
I wish to find all methods that have bytecode length bigger than 8000 bytes (because OOTB java will not optimize it).
I found manual way like this: How many bytes of bytecode has a particular method in Java?
, however my goal is to scan many files automatically.
I tried to use jboss-javassist, but AFAIK getting bytecode length is available only on class level.

Huge methods might indeed never get inlined, however, but I have my doubts regarding the threshold of 8000. This comment suggests a much smaller limit, though it is platform and configuration dependent anyway.
You are right that getting bytecode length needs to process classes on that low level, however, you didn’t specify what actual obstacle you encountered when trying to do that with Javassist. A simple program doing that with Javassist, would be
try(InputStream is=javax.swing.JComponent.class.getResourceAsStream("JComponent.class")) {
ClassFile​ cf = new ClassFile(new DataInputStream(is));
for(MethodInfo mi: cf.getMethods()) {
CodeAttribute ca = mi.getCodeAttribute();
if(ca == null) continue; // abstract or native
int bLen = ca.getCode().length;
if(bLen > 300)
System.out.println(mi.getName()+" "+mi.getDescriptor()+", "+bLen+" bytes");
}
}
This has been written and tested with a recent version of Javassist that uses Generics in the API. If you have a different/older version, you have to use
try(InputStream is=javax.swing.JComponent.class.getResourceAsStream("JComponent.class")) {
ClassFile​ cf = new ClassFile(new DataInputStream(is));
for(Object miO: cf.getMethods()) {
MethodInfo mi = (MethodInfo)miO;
CodeAttribute ca = mi.getCodeAttribute();
if(ca == null) continue; // abstract or native
int bLen = ca.getCode().length;
if(bLen > 300)
System.out.println(mi.getName()+" "+mi.getDescriptor()+", "+bLen+" bytes");
}
}

Construct the stackmap of method while using bcel

I am trying bcel to modify a method by inserting invoke before specific instructions.
It seems that my instrumentation would result in a different stackmap table, which can not be auto-generated by the bcel package itself.
So, my instrumented class file contains the old stackmap table, which would cause error with jvm.
I haved tried with removeCodeAttributes, the method of MethodGen, that can remove all the code attributes. It can work in simple cases, a wrapped function, for example. And it can not work in my case now.
public class Insert{
public static void main(String[] args) throws ClassFormatException, IOException{
Insert isrt = new Insert();
String className = "StringBuilder.class";
JavaClass jclzz = new ClassParser(className).parse();
ClassGen cgen = new ClassGen(jclzz);
ConstantPoolGen cpgen = cgen.getConstantPool();
MethodGen mgen = new MethodGen(jclzz.getMethods()[1], className, cpgen);
InstructionFactory ifac = new InstructionFactory(cgen);
InstructionList ilist = mgen.getInstructionList();
for (InstructionHandle ihandle : ilist.getInstructionHandles()){
System.out.println(ihandle.toString());
}
InstructionFinder f = new InstructionFinder(ilist);
InstructionHandle[] insert_pos = (InstructionHandle[])(f.search("invokevirtual").next());
Instruction inserted_inst = ifac.createInvoke("java.lang.System", "currentTimeMillis", Type.LONG, Type.NO_ARGS, Constants.INVOKESTATIC);
System.out.println(inserted_inst.toString());
ilist.insert(insert_pos[0], inserted_inst);
mgen.setMaxStack();
mgen.setMaxLocals();
mgen.removeCodeAttributes();
cgen.replaceMethod(jclzz.getMethods()[1], mgen.getMethod());
ilist.dispose();
//output the file
FileOutputStream fos = new FileOutputStream(className);
cgen.getJavaClass().dump(fos);
fos.close();
}
}

Removing a StackMapTable is not a proper solution for fixing a wrong StackMapTable. The important cite is:
4.7.4. The StackMapTable Attribute
In a class file whose version number is 50.0 or above, if a method's Code attribute does not have a StackMapTable attribute, it has an implicit stack map attribute (§4.10.1). This implicit stack map attribute is equivalent to a StackMapTable attribute with number_of_entries equal to zero.
Since a StackMapTable must have explicit entries for every branch target, such an implicit StackMapTable will work with branch-free methods only. But in these cases, the method usually doesn’t have an explicit StackMapTable anyway, so you wouldn’t have that problem then (unless the method had branches which your instrumentation removed).
Another conclusion is that you can get away with removing the StackMapTable, if you patch the class file version number to a value below 50. Of course, this is only a solution if you don’t need any class file feature introduced in version 50 or newer…
There was a grace period in which JVMs supported a fall-back mode for class files with broken StackMapTables just for scenarios like yours, where the tool support is not up-to-date. (See -XX:+FailoverToOldVerifier or -XX:-UseSplitVerifier) But the grace period is over now and that support has been declined, i.e. Java 8 JVMs do not support the fall-back mode anymore.
If you want to keep up with the Java development and instrument newer class files which might use features of these new versions you have only two choices:
Calculate the correct StackMapTable manually
Use a tool which supports calculating the correct StackMapTable attributes, e.g. ASM, (see java-bytecode-asm) does support it

Problem validating against an XSD with Java5

I'm trying to validate an Atom feed with Java 5 (JRE 1.5.0 update 11). The code I have works without problem in Java 6, but fails when running in Java 5 with a
org.xml.sax.SAXParseException: src-resolve: Cannot resolve the name 'xml:base' to a(n) 'attribute declaration' component.
I think I remember reading something about the version of Xerces bundled with Java 5 having some problems with some schemas, but i cant find the workaround. Is it a known problem ? Do I have some error in my code ?
public static void validate() throws SAXException, IOException {
List<Source> schemas = new ArrayList<Source>();
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/dc.xsd")));
// Lookup a factory for the W3C XML Schema language
SchemaFactory factory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
// Compile the schemas.
Schema schema = factory.newSchema(schemas.toArray(new Source[schemas.size()]));
Validator validator = schema.newValidator();
// load the file to validate
Source source = new StreamSource(AtomValidator.class.getResourceAsStream("/sample-feed.xml"));
// check the document
validator.validate(source);
}
Update : I tried the method below, but I still have the same problem if I use Xerces 2.9.0. I also tried adding xml.xsd to the list of schemas (as xml:base is defined in xml.xsd) but this time I have
Exception in thread "main" org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document 'null', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Update 2: I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works. I'll try to post a "real answer" from home.
and sorry, I cant reply as a comment : because of security reasons XHR is disabled from work ...

Indeed, people have been mentioning the Java 5 Sun provided SchemaFactory is giving troubles.
So: did you include Xerces in your project yourself?
After including Xerces, you need to ensure it is being used. If you like to hardcode it (well, as a minimal requirement you'd probably use some application properties file to enable and populate the following code):
String schemaFactoryProperty =
"javax.xml.validation.SchemaFactory:" + XMLConstants.W3C_XML_SCHEMA_NS_URI;
System.setProperty(schemaFactoryProperty,
"org.apache.xerces.jaxp.validation.XMLSchemaFactory");
SchemaFactory factory =
SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Or, if you don't want to hardcode, or when your troublesome code would be in some 3rd party library that you cannot change, set it on the java command line or environment options. For example (on one line of course):
set JAVA_OPTS =
"-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema
=org.apache.xerces.jaxp.validation.XMLSchemaFactory"
By the way: apart from the Sun included SchemaFactory implementation giving trouble (something like com.sun.org.apache.xerces.internal.jaxp.validation.xs.schemaFactoryImpl), it also seems that the "discovery" of non-JDK implementations fails in that version. If I understand correctly than, normally, just including Xerces would in fact make SchemaFactory#newInstance find that included library, and give it precedence over the Sun implementation. To my knowledge, that fails as well in Java 5, making the above configuration required.

I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works.
Ah, I didn't realize that xml.xsd is in fact the one referenced as http://www.w3.org/2001/xml.xsd or something like that. That should teach us to always show some XML and XSD fragments as well. ;-)
So, am I correct to assume that 1.) to fix the Java 5 issue, you still needed to include Xerces and set the system property, and that 2.) you did not have xml.xsd available locally?
Before you found your solution, did you happen to try using getResource rather than getResourceAsStream, to see if the exception would then have showed you some more details?
If you actually did have xml.xsd available (so: if getResource did in fact yield a URL) then I wonder what Xerces was trying to fetch from the internet then. Or maybe you did not add that schema to the list prior to adding your own schemas? The order is important: dependencies must be added first.
For whoever gets tot his question using the search: maybe using a custom EntityResolver could have indicated the source of the problem as well (if only writing something to the log and just returning null to tell Xerces to use the default behavior).

Hmmm, just read your "comment" -- editing does not alert people for new replies, so time to ask your boss for some iPhone or some other gadget that is connected to the net directly ;-)
Well, I assume you added:
schemas.add(
new StreamSource(AtomValidator.class.getResourceAsStream("/xml.xsd")));
If so, is xml.xsd actually to be found on the classpath then? I wonder if the getResourceAsStream did not yield null in your case, and how new StreamSource(null) would act then.
Even if getResourceAsStream did not yield null, the resulting StreamSource would still not know where it was loaded from, which may be a problem when trying to include references. So, what if you use the constructor StreamSource(String systemId) instead:
schemas.add(new StreamSource(AtomValidator.class.getResource("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResource("/dc.xsd")));
You might also use StreamSource(InputStream inputStream, String systemId), but I don't see any advantage over the above two lines. However, the documentation explains why passing the systemId in either of the 2 constructors seems good:
This constructor allows the systemID to be set in addition to the input stream, which allows relative URIs to be processed.
Likewise, setSystemId(String systemId) explains a bit:
The system identifier is optional if there is a byte stream or a character stream, but it is still useful to provide one, since the application can use it to resolve relative URIs and can include it in error messages and warnings (the parser will attempt to open a connection to the URI only if there is no byte stream or character stream specified).
If this doesn't work out, then maybe some custom error handler can give you more details:
ErrorHandlerImpl errorHandler = new ErrorHandlerImpl();
validator.setErrorHandler(errorHandler);
:
:
validator.validate(source);
if(errorHandler.hasErrors()){
LOG.error(errorHandler.getMessages());
throw new [..];
}
if(errorHandler.hasWarnings()){
LOG.warn(errorHandler.getMessages());
}
...using the following ErrorHandler to capture the validation errors and continue parsing as far as possible:
import org.xml.sax.helpers.DefaultHandler;
private class ErrorHandlerImpl extends DefaultHandler{
private String messages = "";
private boolean validationError = false;
private boolean validationWarning = false;
public void error(SAXParseException exception) throws SAXException{
messages += "Error: " + exception.getMessage() + "\n";
validationError = true;
}
public void fatalError(SAXParseException exception) throws SAXException{
messages += "Fatal: " + exception.getMessage();
validationError = true;
}
public void warning(SAXParseException exception) throws SAXException{
messages += "Warn: " + exception.getMessage();
validationWarning = true;
}
public boolean hasErrors(){
return validationError;
}
public boolean hasWarnings(){
return validationWarning;
}
public String getMessages(){
return messages;
}
}