Install Apache Arrow Java in Eclipse - java

I'm currently trying to install Apache Arrow for Java in Eclipse and having some troubles.
I've found the Java Packages on https://search.maven.org/search?q=g:org.apache.arrow%20AND%20v:0.17.1
Because I didn't find any information about the installation, I just downloaded all the jar files:
arrow-peformance-0.17.1.jar
arrow-algorithm-0.17.1.jar
arrow-avro-0.17.1.jar
flight-grpc-0.17.1.jar
flight-core-0.17.1.jar
arrow-plasma-0.17.1.jar
arrow-jdbc-0.17.1.jar
arrow-tools-0.17.1.jar
arrow-vector-0.17.1.jar
arrow-memory-0.17.1.jar
arrow-format-0.17.1.jar
I then created a new Java project in Eclipse and copied all the jar files into a folder 'lib' in said project. Then, I added them under Project -> Properties -> Java Build Path -> Libraries -> Add JARs and selected all the jars in my 'lib' folder and applied.
After that, I tried to run the following java-code which I found on https://github.com/animeshtrivedi/blog/blob/master/post/2017-12-26-arrow.md:
import org.apache.arrow.vector.types.pojo.*;
public class FieldTesting {
public static void main(String[] args) {
Field intField = new Field("int", FieldType.nullable(new ArrowType.Int(32, true)), null);
}
}
I get the following error message:
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at org.apache.arrow.vector.types.pojo.Field.<clinit>(Field.java:55)
at FieldTesting.main(FieldTesting.java:6)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 2 more
When I go to Referenced Libraries -> arrow-vector-0.17.1.jar -> org.apache.arrow.vector.types.pojo -> Field.class, it says "Source not found" as seen below. Do I need to attach the source manually?
On the Apache Arrow website, where I downloaded the jar files, there were also "...sources.jar" files which I could download. Do I need to also download those and add them to the "Java Build Path"? (That seems like a lot of work).
There was also a "arrow-java-root" zip folder, can I somehow use this to add the library?
I'm not familiar with adding libraries to Java, so I don't know how to fix this. Any help is appreciated.
(I am on Windows 10, java 8)
Solution:
I converted my Project to a Maven Project (Configure -> Convert to Maven Project) and added the dependency "arrow-java-root". (https://stackoverflow.com/a/26350902) Thanks to andrewjames for pointing this out.
However, I now get the following message when I run the program:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
But the program still terminates correctly, because if I add:
System.out.println(intField.toString())
It prints the field after the message above.

The message you get is because you don't have a log4j2 configuration file, and the logger is warning you about this, so it is going to use a default one, if you want to get rid of that message, you can create a log4j2.xml file in your src/main/resources folder with the following
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
you can read more about configuring log4j here

Related

Gradle set to daemon project base

I am using gradle 6.3 and java 11.
When I run the gradle project see the following lines:
Project base dir set to: /Users/ranjith/.gradle/daemon/6.1
And the checkstyle plugin is pointing to the base directory of the project:
<module name="SuppressionFilter">
<property name="file" value=".mycmp/config/checkstyle/suppressions.xml"/>
</module>
I see the following error when I run the project:
Caused by: java.io.FileNotFoundException:
/Users/ranjith/.gradle/daemon/6.3/.mycmp/config/checkstyle/suppressions.xml
The same works with java 8 though.
JDK 11 expects full path. The issue is described here https://github.com/gradle/gradle/issues/8286#issuecomment-456940831

jooq 3.12.0 codegen tool strategy class not found

I am having trouble getting the jooq codegen tool to run with my custom JooqCodegenStrategy class. I downloaded jooq 3.12.0 and unzipped it into a directory on my Ubuntu system. In this directory, I have the following files.
jooq-3.12.0.jar
jooq-checker-3.12.0.jar
jooq-codegen-3.12.0.jar
jooq-codegen-maven-3.12.0.jar
JooqCodegenStrategy.class
JooqCodegenStrategy.java
jooq-meta-3.12.0.jar
jooq-meta-extensions-3.12.0.jar
jooq-scala_2.12-3.12.0.jar
jooq_sd3.jar
jooq.xml
mysql-connector-java-5.1.47-bin.jar*
reactive-streams-1.0.3.jar
I can successfully run the code generation tool when I do not have the following (below) in my jooq.xml.
<generator>
<strategy>
<name>com.simpletest.JooqCodegenStrategy</name>
</strategy>
</generator>
In order to build the JooqCodegenStrategy class, from terminal, I compile this Java file using:
javac -classpath jooq-meta-3.12.0.jar:jooq-codegen-3.12.0.jar JooqCodegenStrategy.java
That appears to work and it creates the JooqCodegenStrategy.class file. I package that file into a jar so I can reference it on the classpath when running the codegen tool. I create the jar file using:
jar -cvf jooq_sd3.jar JooqCodegenStrategy.class
At this point, I believe I have what I need to run the codegen tool. I run the codegen tool using this:
java -classpath jooq-3.12.0.jar:jooq-meta-3.12.0.jar:jooq-codegen-3.12.0.jar:mysql-connector-java-5.1.47-bin.jar:reactive-streams-1.0.3.jar:jooq_sd3.jar:. org.jooq.codegen.GenerationTool /jooq.xml
The error I get looks like this:
Aug 30, 2019 4:44:12 PM org.jooq.tools.JooqLogger info
INFO: Initialising properties : /jooq.xml
Aug 30, 2019 4:44:13 PM org.jooq.tools.JooqLogger error
SEVERE: Cannot read /jooq.xml. Error : com.simpletest.JooqCodegenStrategy
java.lang.ClassNotFoundException: com.simpletest.JooqCodegenStrategy
at java.net.URLClassLoader.findClass(URLClassLoader.java:382
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.jooq.codegen.GenerationTool.loadClass0(GenerationTool.java:925)
at org.jooq.codegen.GenerationTool.loadClass(GenerationTool.java:869)
at org.jooq.codegen.GenerationTool.run0(GenerationTool.java:380)
at org.jooq.codegen.GenerationTool.run(GenerationTool.java:221)
at org.jooq.codegen.GenerationTool.generate(GenerationTool.java:216)
at org.jooq.codegen.GenerationTool.main(GenerationTool.java:188)
My jooq.xml file looks like this.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<jdbc>
<driver>com.mysql.jdbc.Driver</driver>
<url>jdbc:mysql://localhost:3306/sd3?useSSL=false</url>
<user>myuser</user>
<password>mypass</password>
</jdbc>
<generator>
<name>org.jooq.codegen.JavaGenerator</name>
<strategy>
<name>com.simpletest.JooqCodegenStrategy</name>
</strategy>
<database>
<name>org.jooq.meta.mysql.MySQLDatabase</name>
<inputSchema>sd3</inputSchema>
<includes>.*</includes>
<excludes>sometableshere</excludes>
<forcedTypes>
<forcedType>
<name>JsonElement</name>
<expression>board_data</expression>
<types>JSON</types>
</forcedType>
</forcedTypes>
</database>
<target>
<packageName>model.data</packageName>
<directory>./src/main/java/com/simpletest/domain/</directory>
</target>
<generate>
<relations>true</relations>
<deprecated>true</deprecated>
<instanceFields>true</instanceFields>
<generatedAnnotation>true</generatedAnnotation>
<records>true</records>
<pojos>true</pojos>
<immutablePojos>false</immutablePojos>
<interfaces>false</interfaces>
<daos>true</daos>
<jpaAnnotations>true</jpaAnnotations>
<validationAnnotations>false</validationAnnotations>
<springAnnotations>true</springAnnotations>
<globalObjectReferences>true</globalObjectReferences>
<fluentSetters>true</fluentSetters>
</generate>
</generator>
</configuration>
Any ideas what I am doing wrong?
Thanks for the help.
If compiling your Java class in the local directory works, that means that your class doesn't have a package (i.e. it is in the default package). Which means your qualification of that class is wrong:
<generator>
<strategy>
<name>com.simpletest.JooqCodegenStrategy</name>
</strategy>
</generator>
It is not in the com.simpletest package. The use of the default package is generally discouraged in Java. In many cases (e.g. in this one too, I think), you will not be able to load the class from a random position.
Better add the package declaration to your class:
package com.simpletest;
And move the class in a com/simpletest subdirectory.

GWT compile unable to load class

I'm trying to compile GWT from my project from command line (no Ant/Maven)
like this
java -cp "/home/user/gwt-2.8.2/*:/home/user/myapp/src:/home/user/myapp/lib/*" com.google.gwt.dev.Compiler com.myapp.webservices.SpwrWebServices
but error is thrown
Loading inherited module 'com.myapp.webservices.SpwrWebServices'
Loading inherited module 'gwt.material.design.GwtMaterialWithJQuery'
Loading inherited module 'gwt.material.design.GwtMaterialDesignBase'
[WARN] Line 40: Setting configuration property named 'CssResource.legacy' in module 'gwt.material.design.GwtMaterialDesignBase' that has not been previously defined
[ERROR] Line 18: Unable to load class 'com.myapp.webservices.client.util.GitCommitGenerator'
java.lang.ClassNotFoundException: com.myapp.webservices.client.util.GitCommitGenerator
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.google.gwt.dev.cfg.ModuleDefSchema$ClassAttrCvt.convertToArg(ModuleDefSchema.java:899)
at com.google.gwt.dev.util.xml.HandlerArgs.convertToArg(HandlerArgs.java:64)
at com.google.gwt.dev.util.xml.HandlerMethod.invokeBegin(HandlerMethod.java:221)
at com.google.gwt.dev.util.xml.ReflectiveParser$Impl.startElement(ReflectiveParser.java:296)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.startElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at com.google.gwt.dev.util.xml.ReflectiveParser$Impl.parse(ReflectiveParser.java:349)
at com.google.gwt.dev.util.xml.ReflectiveParser$Impl.access$200(ReflectiveParser.java:70)
at com.google.gwt.dev.util.xml.ReflectiveParser.parse(ReflectiveParser.java:431)
at com.google.gwt.dev.cfg.ModuleDefLoader.nestedLoad(ModuleDefLoader.java:316)
at com.google.gwt.dev.cfg.ModuleDefLoader.load(ModuleDefLoader.java:243)
at com.google.gwt.dev.cfg.ModuleDefLoader.doLoadModule(ModuleDefLoader.java:193)
at com.google.gwt.dev.cfg.ModuleDefLoader.loadFromResources(ModuleDefLoader.java:151)
at com.google.gwt.dev.cfg.ModuleDefLoader.loadFromClassPath(ModuleDefLoader.java:126)
at com.google.gwt.dev.Compiler.compile(Compiler.java:139)
at com.google.gwt.dev.Compiler$1.run(Compiler.java:118)
at com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:55)
at com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:50)
at com.google.gwt.dev.Compiler.main(Compiler.java:125)
I do have the file src/com/myapp/webservices/client/util/GitCommitGenerator.java so I don't understand my this error is showed. My gwt config xml file looks like this
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//Google Inc.//DTD Google Web Toolkit 2.7.0//EN" "\s">
<module rename-to= "SpwrWebServices">
<inherits name="com.google.gwt.user.User" />
<!-- GwtMaterialWithJQuery module is smaller and should be used for production code -->
<!-- <inherits name="gwt.material.design.GwtMaterialWithJQuery" /> -->
<!-- GWTMaterialWithJQueryDebug This module is larger and includes non-minified source for easier debugging -->
<!-- GWT Material -->
<inherits name="gwt.material.design.GwtMaterialWithJQuery"/>
<inherits name="gwt.material.design.addins.GwtMaterialAddins"/>
<inherits name="org.fusesource.restygwt.RestyGWT" />
<inherits name='com.googlecode.gwt.charts.Charts'/>
<inherits name="org.moxieapps.gwt.uploader.Uploader"/>
<set-property name="gwt.logging.enabled" value="TRUE"/>
<set-property name="gwt.logging.logLevel" value="INFO"/>
<generate-with class="com.myapp.webservices.client.util.GitCommitGenerator">
<when-type-assignable class="com.myapp.webservices.client.util.GitCommit"/>
</generate-with>
<source path="client" />
<!-- <source path ="shared" /> -->
<entry-point
class="com.myapp.webservices.client.SpwrWsEntryPoint" />
<set-property name="user.agent" value="gecko1_8,safari"/>
<set-configuration-property name='xsiframe.failIfScriptTag' value='FALSE'/>
</module>
What is wrong with the configuration above and how can I get a clean compilation?
First, generators should not be written new any more, you should start with Java Annotation Processors or some other feature which does not rely on the GWT compiler generating code.
Second, any Generator you write should not be put in your client package, but should be somewhere that GWT will not try to compile it to JS sources. Usually people use the convention of rebind for the classes which let you change bindings of your GWT code and generate new classes.
Finally, it looks like you are running the GWT compiler on your sources without first compiling them with java. First, use javac to compile all your .java files to .class files, and then pass those on the rest of the classpath. Technically you only need your generator and anything it references, but you'd probably like Javac to report errors before GWT does, since it will do a nicer job.
Consider Maven or Gradle to manage your build instead, so it does the javac work for you, and builds the classpath to give to GWT for you too.
The project defines a generator inside the module descriptor:
<generate-with class="com.myapp.webservices.client.util.GitCommitGenerator">
<when-type-assignable class="com.myapp.webservices.client.util.GitCommit"/>
</generate-with>
The generator is located under the client-package. GWT will try to compile it, which will pretty sure fail.
Try this:
create a new package besides the client package, call ist rebind
move the GitCommitGenerator to that package.
update the reference inside your module descriptor to
<generate-with class="com.myapp.webservices.rebind.GitCommitGenerator">
<when-type-assignable class="com.myapp.webservices.client.util.GitCommit"/>
</generate-with>
Think, this will solve the problem.

FileNotFoundException during JNLP download

I was looking for a solution of this problem for weeks now, but did not find anything, which really related to my problem / or I already tried all suggested solutions, without any success.
I have a JNLP file, which is downloaded by the javaws properly (I think), but then before launching, it complains about not finding some files in the /tmp directory. Exact error message is:
CouldNotLoadArgumentException[ Could not load file/URL specified: /tmp/tmp_cache8259898691262575141.tmp]
at com.sun.javaws.Main.launchApp(Unknown Source)
at com.sun.javaws.Main.continueInSecureThread(Unknown Source)
at com.sun.javaws.Main.access$000(Unknown Source)
at com.sun.javaws.Main$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: /tmp/tmp_cache8259898691262575141.tmp (Nincs ilyen fájl vagy könyvtár)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at com.sun.javaws.jnl.LaunchDescFactory.buildDescriptor(Unknown Source)
... 5 more
I checked the /tmp, the folder is of course there, and is writable. Funny thing is, that during downloading the tmp_cacheNNNNNN.tmp files are there! They appear one-by-one, as the download progress moves on. However when the download is finished something (javaws?) deletes all of them, and then starts to complain about missing them.
I have JNLP caching turned on (and intend to have it cached locally). I tried with JRE 1.8.0_40, 1.8.0_65, 1.8.0_66, all gives the same result. Same result on windows and linux, both complain about missing files in the temp folder.
The last JRE I managed to start is the 1.8.0_25, and seems to work with 1.8.0_72 (beta). However the downloaded jar files are still not cached! (if I open up the jcontrol application and look into the cache, the JAR files are not there, and I also don't know why).
My JNLP looks like this:
<?xml version="1.0" encoding="utf-8"?>
<jnlp spec="7.0+"
codebase="http://localhost:8080/jbaf-server/jnlp/swing-client"
href="http://localhost:8080/jbaf-server/clientDownload/client.jnlp" >
<information>
<title>Swing Client</title>
<vendor>www.xy.z</vendor>
<description>Swing Client</description>
<description kind="short">Swing Client</description>
<homepage href="www.xy.z"></homepage>
<icon href="images/jnlp-icon.png"/>
<icon kind="splash" href="images/jnlp-splash.png"/>
<offline-allowed/>
</information>
<security>
<all-permissions/>
</security>
<resources>
<j2se version="1.8+" initial-heap-size="256m" max-heap-size="1024m"/>
<property name="jnlp.versionEnabled" value="true" />
<jar href="jbaf-swing-client.jar" version="0.3.0" main="true"/>
.... (lots of jar files)
<jar href="spring-expression.jar" version="4.2.3.RELEASE"/>
<jar href="spring-tx.jar" version="4.2.3.RELEASE"/>
<jar href="commons-codec.jar" version="1.10"/>
</resources>
<application-desc main-class="org.jbaf.swingclient.Main">
<argument>http://localhost:8080/jbaf-server</argument>
<argument>false</argument>
</application-desc>
</jnlp>
The JARs are signed with self-signed cert.
I have no idea why this happens. Anyone faced such a problem?
Finally I managed to find the problem!
It was caused by following HTTP headers set by Tomcat:
Cache-Control
Pragma
Expires
I did not set these headers in my Servlets, but it seems that these were added by some of the filter previously. I set these header values to "" (empty string), and the JDK started to cache my files, and the application was started finally OK!
However if you set the Cache-Control to no-cache, the JDK still fails to start the application. The files are saved to the System/User temp directory, but afterwards it will not start, because the temp files are removed. This was only working with JDK 1.8.0_72 (BETA), all previous version failed!
Long story short: if you have problems with your JNLP where the WebStart tells you not finding files in the temp directory, check your HTTP Response Headers!

is it possible to load jars-in-jars in JWS?

I have found MANY threads on packing all dependencies along with the project into one jar package, and it seems like there are many different ways to achieve this (oneJar, FatJar, Ant-build...)
So cooking up my own recipe, I have (after quite some effort) managed to package the project I am working on. In this one jar file, there is the code for the project plus all the jars that the project depends on which are loaded with jar-in-jar-loader that comes with eclipse. The resultant works fine on a number of different platforms, when it's ran through the terminal invoked via java -jar myjarfile.
Peachy, you might say, here's the problem though; when I sign my jar and try to run it via javaws (which is the ultimate goal here) I get an exception which I have decrypted to mean that libraries (in the case below org.apache.commons.lang.SystemUtils) are unaccessible.
So here's my question; is it not possible to load jars in jars when the applications is deployed for Java Web Start? If it is possible, what am I doing wrong? If not, what's the best alternative?
Thanks!
Below is the JNLP file along with the stackTrace I get when I run javaws myJNLPfile
<?xml version="1.0" encoding="utf-8"?>
<jnlp spec="1.0+"
codebase="file:///home/ukirik/workspace/myproject/dist"
href="project.jnlp">
<information>
<!-- Project info -->
</information>
<security>
<all-permissions />
</security>
<resources os="Mac\ OS\ X">
<j2se version="1.6+" java-vm-args="-XstartOnFirstThread"/>
</resources>
<resources>
<jar href="myjar-jws.jar" />
</resources>
<application-desc main-class="org.gvt.RuntimeMain"/>
</jnlp>
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.javaws.Launcher.executeApplication(Launcher.java:1799)
at com.sun.javaws.Launcher.executeMainClass(Launcher.java:1745)
at com.sun.javaws.Launcher.doLaunchApp(Launcher.java:1507)
at com.sun.javaws.Launcher.run(Launcher.java:129)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/lang/SystemUtils
at org.gvt.RuntimeMain.loadSwtJar(RuntimeMain.java:27)
at org.gvt.RuntimeMain.main(RuntimeMain.java:13)
... 9 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.SystemUtils
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at com.sun.jnlp.JNLPClassLoader.findClass(JNLPClassLoader.java:332)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 11 more
If you are using the Eclipse jar-in-jar loader I think that you may need this in the jnlp file
<application-desc main-class="org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader"/>
That's assuming that your manifest is looking like this...
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 20.1-b02 (Sun Microsystems Inc.)
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
Rsrc-Main-Class: yourapp.mainclass
Rsrc-Class-Path: ./ swt.jar velocity-1.7-dep.jar
Class-Path: .
It seems like the problem is with the classloader.
You may want to use a custom classloader in JWS, as described here:
http://lopica.sourceforge.net/faq.html#customcl
Depending on exactly what you need to do, in terms of packaging, there are a couple of options.
The problem you're running into is that you're passing org.gvt.RuntimeMain as your main class for your jar in your JNPL file, but that's a one-jar jar. Because it's a one-jar jar, you need to provide class com.simontuffs.onejar.Boot instead, as
<application-desc main-class="com.simontuffs.onejar.Boot"/>
The reason for it, is that the one-jar plugin will generate that class, which will make use of a classloader that understands jars within jars, and then invoke your org.gvt.RuntimeMain class (it figures it out through looking at the manifests's One-Jar-Main-Class: header).

Categories

Resources