Many errors while running Renjin with packages in Eclipse - java

I've been trying to run some simple code from the first lesson of R for Data Science using Renjin in Eclipse. The reason I'm using Renjin is because I'm eventually going to develop this into a Java data analysis program. Here is the link to the R for Data Science lesson I'm following: https://r4ds.had.co.nz/data-visualisation.html
I got all my dependencies in the pom.xml file of my Maven project, and when I test my pom.xml file, it builds successfully. Here's my pom.xml just in case:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Tests</groupId>
<artifactId>Test1</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Test1</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.renjin</groupId>
<artifactId>renjin-script-engine</artifactId>
<version>3.5-beta76</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>ggplot2</artifactId>
<version>3.2.0-b8</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>dplyr</artifactId>
<version>0.8.2-b6</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>feather</artifactId>
<version>0.3.1-b9</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>hms</artifactId>
<version>0.4.2-b18</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>stringr</artifactId>
<version>1.4.0-b6</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>lubridate</artifactId>
<version>1.7.4-b4</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>forcats</artifactId>
<version>0.3.0-b9</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>haven</artifactId>
<version>2.0.0-b1</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>httr</artifactId>
<version>1.4.0-b1</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>jsonlite</artifactId>
<version>1.6-b1</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>readxl</artifactId>
<version>1.3.1-b11</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>rvest</artifactId>
<version>0.3.2-b115</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>xml2</artifactId>
<version>1.2-renjin-8</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>modelr</artifactId>
<version>0.1.1-b31</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>broom</artifactId>
<version>0.4.4-b1</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>tidyr</artifactId>
<version>0.8.0-b1</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>readr</artifactId>
<version>1.3.1-b3</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>purrr</artifactId>
<version>0.3.2-b11</version>
</dependency>
<dependency>
<groupId>org.renjin.cran</groupId>
<artifactId>tibble</artifactId>
<version>2.1.3-b9</version>
</dependency>
</dependencies>
<build>
<pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
<plugins>
<!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
<repositories>
<repository>
<id>bedatadriven</id>
<name>bedatadriven public repo</name>
<url>https://nexus.bedatadriven.com/content/groups/public/</url>
</repository>
</repositories>
</project>
However, when I try to run my App.java program with the R for Data Science code, it gives me loads of errors. Here is my App.java:
package Tests.Test1;
import javax.script.*;
import org.renjin.script.*;
import java.awt.*;
import javax.swing.*;
// ... add additional imports here ...
public class App {
public static void main(String[] args) throws Exception {
// create a script engine manager:
RenjinScriptEngineFactory factory = new RenjinScriptEngineFactory();
// create a Renjin engine:
ScriptEngine engine = factory.getScriptEngine();
// ... put your Java code here ...
engine.eval("library(dplyr)");
engine.eval("library(ggplot2)");
engine.eval("library(tidyr)");
engine.eval("library(readr)");
engine.eval("library(purrr)");
engine.eval("library(tibble)");
engine.eval("?mpg");
engine.eval("ggplot(data = mpg) + geom_point(mapping = aes(x = displ, y = hwy))");
}
}
And here's the long list of errors I'm getting:
EEK! colSums.computeMeans() called through getElementAsDouble()
EEK! colSums.computeMeans() called through getElementAsDouble()
EEK! colSums.computeMeans() called through getElementAsDouble()
EEK! colSums.computeMeans() called through getElementAsDouble()
org.renjin.eval.EvalException: Exception initializing compiled GNU R library class org.renjin.cran.dplyr.dplyr
at org.renjin.primitives.packaging.DllInfo.initialize(DllInfo.java:141)
at org.renjin.primitives.packaging.Namespace.loadDynamicLibrary(Namespace.java:383)
at org.renjin.primitives.packaging.Namespace.importDynamicLibrary(Namespace.java:296)
at org.renjin.primitives.packaging.Namespace.initImports(Namespace.java:274)
at org.renjin.primitives.packaging.NamespaceRegistry.load(NamespaceRegistry.java:175)
at org.renjin.primitives.packaging.NamespaceRegistry.getNamespace(NamespaceRegistry.java:143)
at org.renjin.primitives.packaging.NamespaceRegistry.getNamespace(NamespaceRegistry.java:114)
at org.renjin.primitives.packaging.Packages.library(Packages.java:39)
at org.renjin.primitives.R$primitive$library.doApply(R$primitive$library.java:68)
at org.renjin.primitives.R$primitive$library.applyPromised(R$primitive$library.java:33)
at org.renjin.sexp.BuiltinFunction.apply(BuiltinFunction.java:100)
at org.renjin.primitives.special.InternalFunction.apply(InternalFunction.java:46)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.primitives.special.BeginFunction.apply(BeginFunction.java:39)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.sexp.Closure.applyPromised(Closure.java:200)
at org.renjin.sexp.Closure.apply(Closure.java:133)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.sexp.ExpressionVector.eval(ExpressionVector.java:85)
at org.renjin.eval.Context.evaluate(Context.java:280)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:174)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:133)
at Tests.Test1.App.main(App.java:19)
Caused by: java.lang.ArithmeticException: / by zero
at java.base/java.lang.Integer.remainderUnsigned(Integer.java:1564)
at org.renjin.cran.dplyr.hybrid__._ZN5boost9unordered6detail12prime_policyIjE9to_bucketEjj(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZNK5boost9unordered6detail5tableINS1_3mapISaISt4pairIKP7SEXPRECN5dplyr6hybrid15hybrid_functionEEES6_SA_NS_4hashIS6_EESt8equal_toIS6_EEEE14hash_to_bucketEj(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZNK5boost9unordered6detail5tableINS1_3mapISaISt4pairIKP7SEXPRECN5dplyr6hybrid15hybrid_functionEEES6_SA_NS_4hashIS6_EESt8equal_toIS6_EEEE14find_node_implIS6_SG_EEPNS1_8ptr_nodeISB_EEjRKT_RKT0_(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZNK5boost9unordered6detail5tableINS1_3mapISaISt4pairIKP7SEXPRECN5dplyr6hybrid15hybrid_functionEEES6_SA_NS_4hashIS6_EESt8equal_toIS6_EEEE9find_nodeEjRS7_(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZN5boost9unordered6detail5tableINS1_3mapISaISt4pairIKP7SEXPRECN5dplyr6hybrid15hybrid_functionEEES6_SA_NS_4hashIS6_EESt8equal_toIS6_EEEE14emplace_uniqueINS1_13emplace_args1ISB_EEEES4_INS0_15iterator_detail8iteratorINS1_8ptr_nodeISB_EEEEbERS7_RKT_(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZN5boost9unordered13unordered_mapIP7SEXPRECN5dplyr6hybrid15hybrid_functionENS_4hashIS3_EESt8equal_toIS3_ESaISt4pairIKS3_S6_EEE7emplaceISD_EESB_INS0_15iterator_detail8iteratorINS0_6detail8ptr_nodeISD_EEEEbERKT_(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZN5boost9unordered13unordered_mapIP7SEXPRECN5dplyr6hybrid15hybrid_functionENS_4hashIS3_EESt8equal_toIS3_ESaISt4pairIKS3_S6_EEE6insertERKSD_(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZN5dplyr6hybrid11hybrid_initEP7SEXPRECS2_S2_NS0_9hybrid_idE(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._ZN5dplyr6hybrid4initEv(hybrid.cpp)
at org.renjin.cran.dplyr.hybrid__._Z22init_hybrid_inline_mapP8_DllInfo(hybrid.cpp)
at org.renjin.cran.dplyr.RcppExports__.R_init_dplyr(RcppExports.cpp)
at org.renjin.cran.dplyr.dplyr.R_init_dplyr(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.renjin.primitives.packaging.DllInfo.initialize(DllInfo.java:137)
... 22 more
org.renjin.eval.EvalException: Exception initializing compiled GNU R library class org.renjin.cran.tidyr.tidyr
at org.renjin.primitives.packaging.DllInfo.initialize(DllInfo.java:141)
at org.renjin.primitives.packaging.Namespace.loadDynamicLibrary(Namespace.java:383)
at org.renjin.primitives.packaging.Namespace.importDynamicLibrary(Namespace.java:296)
at org.renjin.primitives.packaging.Namespace.initImports(Namespace.java:274)
at org.renjin.primitives.packaging.NamespaceRegistry.load(NamespaceRegistry.java:175)
at org.renjin.primitives.packaging.NamespaceRegistry.getNamespace(NamespaceRegistry.java:143)
at org.renjin.primitives.packaging.NamespaceRegistry.getNamespace(NamespaceRegistry.java:114)
at org.renjin.primitives.packaging.Packages.library(Packages.java:39)
at org.renjin.primitives.R$primitive$library.doApply(R$primitive$library.java:68)
at org.renjin.primitives.R$primitive$library.applyPromised(R$primitive$library.java:33)
at org.renjin.sexp.BuiltinFunction.apply(BuiltinFunction.java:100)
at org.renjin.primitives.special.InternalFunction.apply(InternalFunction.java:46)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.primitives.special.BeginFunction.apply(BeginFunction.java:39)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.sexp.Closure.applyPromised(Closure.java:200)
at org.renjin.sexp.Closure.apply(Closure.java:133)
at org.renjin.sexp.FunctionCall.eval(FunctionCall.java:80)
at org.renjin.sexp.ExpressionVector.eval(ExpressionVector.java:85)
at org.renjin.eval.Context.evaluate(Context.java:280)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:174)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:133)
at Tests.Test1.App.main(App.java:21)
Caused by: java.lang.NoSuchFieldError: _ZTVN10__cxxabiv120__si_class_type_infoE
at org.renjin.cran.tidyr.RcppExports__._ZTIN4Rcpp9exceptionE$$clinit(RcppExports.cpp)
at org.renjin.cran.tidyr.RcppExports__.<clinit>(RcppExports.cpp)
at org.renjin.cran.tidyr.tidyr.R_init_tidyr(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.renjin.primitives.packaging.DllInfo.initialize(DllInfo.java:137)
... 22 more
I'm not quite sure how to get the code to work. If you have any ideas on how to fix this, I'd love to hear them. Please keep in mind that I am a beginner though, and I may need things in simpler/more thorough terms. If you need any more information, let me know, and I'll respond as soon as possible. Any and all help is appreciated. Thank you.

Related

Package accessible from more than one module in Intellij

I'm trying to use google text to speech but there is a problem with the imports:
import com.google.cloud.texttospeech.v1beta1.AudioConfig;
import com.google.cloud.texttospeech.v1beta1.AudioEncoding;
import com.google.cloud.texttospeech.v1beta1.SsmlVoiceGender;
import com.google.cloud.texttospeech.v1beta1.SynthesisInput;
import com.google.cloud.texttospeech.v1beta1.SynthesizeSpeechResponse;
import com.google.cloud.texttospeech.v1beta1.TextToSpeechClient;
import com.google.cloud.texttospeech.v1beta1.VoiceSelectionParams;
It shows various errors where it's using the same modules from different packages:
java: module google.cloud.texttospeech reads package com.google.cloud.texttospeech.v1beta1 from both google.cloud.texttospeech and proto.google.cloud.texttospeech.v1beta1
I have these requirements in my module-info file:
requires proto.google.cloud.texttospeech.v1beta1;
requires google.cloud.texttospeech;
I also get a yellow warning with these requirements:
Name of automatic module 'proto.google.cloud.texttospeech.v1beta1' is unstable, it is derived from the module's file name.
pom.xml is as follows (removed project tag for clarity):
<modelVersion>4.0.0</modelVersion>
<groupId>com.cloze</groupId>
<artifactId>cloze</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>26.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>18.0.1</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-media</artifactId>
<version>18.0.1</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-texttospeech</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<release>11</release>
</configuration>
</plugin>
<plugin>
<groupId>org.openjfx</groupId>
<artifactId>javafx-maven-plugin</artifactId>
<version>0.0.6</version>
<executions>
<execution>
<!-- Default configuration for running -->
<!-- Usage: mvn clean javafx:run -->
<id>default-cli</id>
<configuration>
<mainClass>com.cloze.App</mainClass>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Contact the maintainers of the google-cloud-texttospeech library and ask them to provide documentation (and perhaps an updated implementation) for using their library from a modular Java app.
In the meantime go to openjfx.io and review documentation on non-modular JavaFX apps and try following it.
Asker writes:
If I remove the module-info file to make it non modular that does indeed solve this problem.

Why is a java.lang.NoClassDefFoundError raised when I run my JAR with Apache Storm ? (org/apache/storm/kafka/spout/KafkaSpoutConfig)

I started last week a formation in Kafka and Storm at OpenClassRooms. During practical work, I encounter an error when I try to execute a JAR containing my java code for Storm.
No problem when compiling the project in Java, no problem when packaging with maven, the problem only occurs when running the JAR
theirman#vm-debian:/data/eclipse-workspace/velos$ storm jar target/velos-1.0-SNAPSHOT.jar velos.App remote
Running: /usr/lib/jvm/java/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/apps/storm -Dstorm.log.dir=/apps/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file= -cp /apps/storm/*:/apps/storm/lib/*:/apps/storm/extlib/*:target/velos-1.0-SNAPSHOT.jar:/apps/storm/conf:/apps/storm/bin: -Dstorm.jar=target/velos-1.0-SNAPSHOT.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} velos.App remote
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/storm/kafka/spout/KafkaSpoutConfig
at velos.App.main(App.java:22)
Caused by: java.lang.ClassNotFoundException: org.apache.storm.kafka.spout.KafkaSpoutConfig
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 1 more
App.java
package velos;
import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.generated.StormTopology;
import org.apache.storm.kafka.spout.KafkaSpout;
import org.apache.storm.kafka.spout.KafkaSpoutConfig;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseWindowedBolt;
import org.apache.storm.tuple.Fields;
public class App
{
public static void main( String[] args ) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException
{
TopologyBuilder builder = new TopologyBuilder();
KafkaSpoutConfig.Builder<String, String> spoutConfigBuilder = KafkaSpoutConfig.builder("localhost:9092", "velib-stations");
spoutConfigBuilder.setGroupId("city-stats");
KafkaSpoutConfig<String, String> spoutConfig = spoutConfigBuilder.build();
builder.setSpout("stations", new KafkaSpout<String, String>(spoutConfig));
builder.setBolt("station-parsing", new StationParsingBolt()).shuffleGrouping("stations");
builder.setBolt("city-stats", new CityStatsBolt().withTumblingWindow(BaseWindowedBolt.Duration.of(1000*60*5))).fieldsGrouping("station-parsing", new Fields("city"));
builder.setBolt("save-results", new SaveResultsBolt()).fieldsGrouping("city-stats", new Fields("city"));
StormTopology topology = builder.createTopology();
Config config = new Config();
config.setMessageTimeoutSecs(60*30);
String topologyName = "velos";
if(args.length > 0 && args[0].equals("remote")) {
StormSubmitter.submitTopology(topologyName, config, topology);
}
else
{
LocalCluster cluster = new LocalCluster();
cluster.submitTopology(topologyName, config, topology);
}
}
}
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>velos</groupId>
<artifactId>velos</artifactId>
<version>1.0-SNAPSHOT</version>
<name>velos</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>1.1.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.12</artifactId>
<version>0.10.2.0</version>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>1.0.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka-client</artifactId>
<version>1.1.0</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
<plugins>
<!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
storm-kafka or storm-kafka-client likely are not provided on the storm classpath, so you would need to remove the scope from those
Then you will also want to try shading your JAR so that all dependencies are available at runtime

ClassNotFoundException when running fat JAR configured by spring-boot-maven-plugin

I am trying to create executable JAR file for my spring boot application.
To achieve this spring-boot-maven-plugin has been used with main class specified and packaging to jar.
Unfortunately after running output JAR file i receive java.lang.ClassNotFoundException pointing to my main class.
Below is my pom.xml and main class. Thanks in advance for help.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.test</groupId>
<artifactId>interflight</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>interflight</name>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.9.RELEASE</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<feign-gson.version>7.2.1</feign-gson.version>
<spring-cloud.version>Edgware.SR1</spring-cloud.version>
<gson.version>2.8.0</gson.version>
<assertj-core.version>3.8.0</assertj-core.version>
<startClass>com.test.interflight.configuration.InterflightApplication</startClass>
</properties>
<dependencies>
<!--External dependencies-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-feign</artifactId>
</dependency>
<dependency>
<groupId>com.netflix.feign</groupId>
<artifactId>feign-gson</artifactId>
<version>${feign-gson.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<!--Test dependencies-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>${assertj-core.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<finalName>${project.artifactId}</finalName>
<layout>ZIP</layout>
<mainClass>com.test.interflight.configuration.InterflightApplication</mainClass>
</configuration>
</plugin>
</plugins>
</build>
</project>
Output error:
$ java -jar interflight.jar
Exception in thread "main" java.lang.ClassNotFoundException: com.test.interfl
ight.configuration.InterflightApplication
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(Laun
chedURLClassLoader.java:94)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner
.java:46)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLau
ncher.java:587)
Main class:
#SpringBootApplication
#ComponentScan(basePackages = {"com.test.interflight.controller", "com.test.interflight.implementation.*"})
#EnableFeignClients(basePackages = {"com.test.interflight.api.restclient"},
defaultConfiguration = FeignConfiguration.class)
public class InterflightApplication {
public static void main(String[] args) {
SpringApplication.run(InterflightApplication.class, args);
}
}
Short answer:
Looks like you're using the ZIP layout when you should be using the JAR layout. Change <layout>ZIP</layout> to <layout>JAR</layout> in your Spring Boot maven plugin declaration.
Longer answer:
Take a look at the docs which describe the ZIP layout as:
ZIP (alias to DIR): similar to the JAR layout using
PropertiesLauncher.
And of which the PropertiesLauncher is described in the docs as:
Launcher for archives with user-configured classpath and main class
via a properties file...
Looks in various places for a properties file to extract loader settings, defaulting to application.properties... No default, but will fall back to looking for a Start-Class in a MANIFEST.MF
If you want it to be user specified in your Jar file, specify (or check and see if you have one already) in the meta-inf folder of your Jar, or specify the property yourself.
After several days dealing with various plugins, finally I could make a fat jar (spring boot) with externalize config files everywhere I wanted to be. You can use the following plugins:"config" is the path for your config files next to the jar file.
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<finalName>jarfilename</finalName>
<mainClass>com.start.YourStarter</mainClass>
<layout>ZIP</layout>
</configuration>
<executions>
<execution>
<phase>install</phase>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/log4j.xml</exclude>
<exclude>deploy/**</exclude>
<exclude>**/*.properties</exclude>
<exclude>**/*.txt</exclude>
</excludes>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>com.start.YourStarter</mainClass>
<useUniqueVersions>false</useUniqueVersions>
</manifest>
<manifestEntries>
<Class-Path>config/</Class-Path>
</manifestEntries>
</archive>
</configuration>
</plugin>

package org.apache.hadoop.hive.ql.exec.vector does not exist

I'm trying to work with Hive UDF's and UDAF's and so far, I've managed quite well in doing so using maven from the command line on a VM that has essentially been set up quite well by someone very knowledgable. I'm new to Java (projects, not so much the language) but I can at least see what my current issue is. What I don't know is why it's happening and how to fix it.
So...
my problem is that for whatever reason, when running
mvn install
the compiler can't find any classes past
/org/apache/hadoop/hive/ql/exec
It can find all classes in the 'exec' folder, but nothing in any subfolders nested in 'exec'. I don't understand why this is the case. Here is my POM file.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.8</version>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.matthewrathbone.example.RawMapreduce</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
<modelVersion>4.0.0</modelVersion>
<groupId>com.matthewrathbone.example</groupId>
<artifactId>hive-extensions</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>hive-extensions</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.0.0-mr1-cdh4.3.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>0.10.0-cdh4.3.1</version>
<scope>provided</scope>
</dependency>
<!-- TEST DEPENDENCIES -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-test</artifactId>
<version>2.0.0-mr1-cdh4.1.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<repositories>
<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
</repositories>
</project>
An example of one of the many errors I get is in the title. 'vector' is a folder located in /org/apache/hadoop/hive/ql/exec.
As far as I can tell, the hive-exec.jar file is listed as a dependency in my POM file, so I'm not really sure why the compiler is having trouble locating all the classes contained within it (it can find some, as mentioned previously).
Essentially what I'm trying to do is incorporate one of the built-in UDAF's as my own temporary function, so that I can bypass Hive's restrictions on sub-querying inside the 'SELECT' statement.
So, to be more specific, I made some adjustments and eventually narrowed it down to compiler was looking for class VectorizedUDAFs which as it turns out, my version of hive-exec .jar does not have.... go figure. Thus, it couldn't find it because it actually did not exist...

Maven EAR multimodule project not packaging persistence.xml

I`m working in a EAR project with Maven which has 2 modules. Images speaks louder than words, so let me show you the structure:
Parent pom project and modules
sigea-model contains model, repository and service layers (The "M" in MVC). sigea-web contains web pages and controller beans (VC) and sigea-ear is just a wrapper to package the other 2 modules in a EAR package.
Configuration files in modules
As you can see, sigea-ear has an empty META-INF folder. Both beans.xml files in sigea-model and sigea-web are just empty marker files because AFAIK, CDI by default search in all annotated classes (but this is not the problem right now). persistence.xml is a simple file which uses JTA transactions with a connection pool (which is working because I ping from the Glassfish's admin console and is successful).
Finally, when I package the application I get the following:
As you can see, there's no persistence.xml. All this came out because I deployed the application successfully but in the first click I got the Exception
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean
...
Caused by: java.lang.IllegalStateException: Unable to retrieve EntityManagerFactory for unitName null
Here are my pom files:
pom.xml[sigea-app] (parent project)
<project ...>
<modelVersion>4.0.0</modelVersion>
<groupId>ar.edu.unt.sigea</groupId>
<artifactId>sigea-app</artifactId>
<version>1.0</version>
<packaging>pom</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<modules>
<module>sigea-model</module>
<module>sigea-web</module>
<module>sigea-ear</module>
</modules>
<build>
<pluginManagement>
...
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencyManagement>
<!-- I suppress some lines for brevity -->
<dependencies>
<dependency>
<artifactId>sigea-model</artifactId>
</dependency>
<dependency>
<artifactId>sigea-model</artifactId>
<type>ejb</type>
</dependency>
<dependency>
<artifactId>sigea-model</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<artifactId>sigea-web</artifactId>
<type>war</type>
</dependency>
<dependency>
<artifactId>sigea-web</artifactId>
<type>pom</type>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
...
</dependencies>
</project>
pom.xml[sigea-ear]
<project ...>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>ar.edu.unt.sigea</groupId>
<artifactId>sigea-app</artifactId>
<version>1.0</version>
</parent>
<artifactId>sigea-ear</artifactId>
<packaging>ear</packaging>
<dependencies>
<dependency>
<artifactId>sigea-model</artifactId>
<type>ejb</type>
</dependency>
<dependency>
<artifactId>sigea-web</artifactId>
<type>war</type>
</dependency>
<dependency>
<artifactId>sigea-web</artifactId>
<type>pom</type>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-ear-plugin</artifactId>
<configuration>
<defaultLibBundleDir>lib/</defaultLibBundleDir>
<skinnyWars>true</skinnyWars>
<modules>
<webModule>
<groupId>${project.groupId}</groupId>
<artifactId>sigea-web</artifactId>
<contextRoot>/sigea</contextRoot>
</webModule>
<ejbModule>
<groupId>${project.groupId}</groupId>
<artifactId>sigea-model</artifactId>
</ejbModule>
</modules>
</configuration>
</plugin>
</plugins>
</build>
</project>
pom.xml[sigea-web]
<project ...>
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>sigea-app</artifactId>
<groupId>ar.edu.unt.sigea</groupId>
<version>1.0</version>
</parent>
<groupId>ar.edu.unt.sigea</groupId>
<artifactId>sigea-web</artifactId>
<version>1.0</version>
<packaging>war</packaging>
<name>sigea-web</name>
<dependencies>
<!-- Some dependencies including sigea-model -->
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<packagingExcludes>WEB-INF/lib/*.jar</packagingExcludes>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
pom.xml[sigea-model] is not important as it just defines some dependencies for test and is configured to generate a package with the test classes, which are used in sigea-web for test purposes also.
Finally the question: What's failing in my configuration that doesn't package the persistence.xml file? If that's not the problem for the IllegalStateException with the message shown above: What are posible causes for that exception?
Thanks in advance for your answers.
I solved the problem by changing a dependency. In sigea-model I had
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>5.1.0.Final</version>
<scope>test</scope>
</dependency>
Which I used to manage the persistence context for my test methods. I didnt inquire very much in the Glassfish JPA provided implementation but maybe it's Eclipse Link instead of Hibernate. Apparently there are some incompatibility issue between those libraries. I movedhibernate-entitymanager` to compile scope, like this:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>5.1.0.Final</version>
</dependency>
And now the projects compile with no problem. It was also useful the comment made by khmarbaise, it simplified the project configuration, thank you very much.

Categories

Resources