Develop Reference

Weka Filter class is not found

Writing code on Eclipse with Weka. Trying to use the weka.filters.Filter method but I constantly receive this error:
java.lang.NoClassDefFoundError: weka/filters/Filter
Everything else seems to be imported right and I see the filter class in the Package Exploration.
Thank you to everyone in advance

Haven't used Eclipse in a long time, as I prefer IntelliJ IDEA.
Here is what I did:
Create a new workspace
Create a new Java project
Unchecked Create module-info.java and clicked on Next
Added my external weka.jar under Classpath and clicked on Finish:
Created class Testing.java in package myweka with this content:
package myweka;
import weka.filters.Filter;
import weka.filters.MultiFilter;
public class Testing {
public static void main(String[] args) throws Exception {
Filter f = new MultiFilter();
System.out.println(f);
}
}
Here it is what it looks like in Eclipse:
Executed the class without problems (just outputs the classname in the console).

Instead of simply adding an external jar to your project, you could set up a Maven project. Here are the steps:
Create a new directory to house all your code
Create a file called pom.xml (Maven Project Object Model) and add this content (the libraries that are required for the project are listed inside the dependencies tag, this example uses Weka 3.9.5; the 3.8.5 dependency is commented out; you can only use either or):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>MyGroup</groupId>
<artifactId>MyWeka</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>MyWeka</name>
<description>Project using Weka.</description>
<dependencies>
<!-- Weka 3.9.5 -->
<dependency>
<groupId>nz.ac.waikato.cms.weka</groupId>
<artifactId>weka-dev</artifactId>
<version>3.9.5</version>
</dependency>
<!-- Weka 3.8.5 -->
<!--dependency>
<groupId>nz.ac.waikato.cms.weka</groupId>
<artifactId>weka-stable</artifactId>
<version>3.8.5</version>
</dependency-->
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.3.2</version>
</plugin>
</plugins>
</build>
</project>
Create directory src/main/java/myweka in the same directory as the pom.xml file.
Place the following content as Testing.java in that directory:
package myweka;
import weka.filters.Filter;
import weka.filters.MultiFilter;
public class Testing {
public static void main(String[] args) throws Exception {
Filter f = new MultiFilter();
System.out.println(f);
}
}
Launch Eclipse and create a new workspace.
Import your project
Import the project as an Existing Maven Project
Select the directory where your pom.xml is located and click on Finish.
Once Eclipse has finished the import, you can execute the myweka.Testing class, which will just output the filter's classname in the console.

Multi-module JavaFX maven project packaging issue

This is an attempt to create a multi-module JavaFX application with maven.
Given the following structure of the project:
project
| pom1.xml
|_____ Word Generator (Folder)
| pom2.xml
|_____logic (folder)
| WordGenerator
|_____UI (folder)
| pom3.xml
|_____marty
| App
| PrimaryController
| SecondaryController
We have the following structure of the pom files in order of the scheme above:
pom1.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.games.marty</groupId>
<artifactId>words</artifactId>
<packaging>pom</packaging>
<version>0.1</version>
<modules>
<module>UI</module>
<module>Word Generator</module>
</modules>
<properties>
<maven.compiler.source>16</maven.compiler.source>
<maven.compiler.target>16</maven.compiler.target>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>16</source>
<target>16</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
pom2.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>words</artifactId>
<groupId>org.games.marty</groupId>
<version>0.1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>word.generator</artifactId>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>16</maven.compiler.source>
<maven.compiler.target>16</maven.compiler.target>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>16</source>
<target>16</target>
</configuration>
</plugin>
<plugin>
<!-- Build an executable JAR -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>org.games.marty.logic.WordGenerator</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
pom3.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>UI</artifactId>
<version>0.1</version>
<parent>
<artifactId>words</artifactId>
<groupId>org.games.marty</groupId>
<version>0.1</version>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>16</maven.compiler.source>
<maven.compiler.target>16</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>16</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-fxml</artifactId>
<version>16</version>
</dependency>
<dependency>
<groupId>org.games.marty</groupId>
<artifactId>word.generator</artifactId>
<version>0.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<release>16</release>
</configuration>
</plugin>
<plugin>
<groupId>org.openjfx</groupId>
<artifactId>javafx-maven-plugin</artifactId>
<version>0.0.6</version>
<executions>
<execution>
<!-- Default configuration for running -->
<!-- Usage: mvn clean javafx:run -->
<id>default-cli</id>
<configuration>
<mainClass>org.games.marty.App</mainClass>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<!-- Build an executable JAR -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>org.games.marty.App</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
The way we have attempted to build the application in order for the UI to have access to the WordGenerator logic is to maven package the result from the pom1.xml directive
We get the above error as mentioned earlier:
Error: Could not find or load main class org.games.marty.App
Caused by: java.lang.NoClassDefFoundError: javafx/application/Application
As far as my understanding goes, the JavaFX dependencies are installed throught maven and should be available but they are missing?

Packaging via mvn package using the maven-jar-plugin is not enough
mvn package, by default, is just going to package the jar for your application code, it isn't going to include all of the dependant library code (which is why the dependent code cannot be found when you attempt to run your application).
You could package your application code and dependant libraries using an assembly, as detailed in How can I create an executable JAR with dependencies using Maven?, though that approach is not the only one to solve your problem.
You need to build some kind of runtime image
There are numerous options for building runtime images and I don't know your requirements, so I can't recommend what you should do instead. Example options are:
A zip/tar of your application plus libraries in a separate directory.
The creation of a single jar that includes all dependant code.
Either of solutions 1 or 2, plus the inclusion of a packaged JRE.
A runtime image with your code and libraries which also uses just the custom runtime portions of the JRE and JavaFX modules you need (using jlink).
A native installer for either 3 or 4 (using jpackage + a native installer creation tool, e.g. WIX, RPM, DEB installer creators).
The last method (native installer), is the packaging, distribution, and installation method I would recommend for most non-trivial applications.
You need to research how to do this
To get your solution, you will need to do your own research, and, once you have chosen an approach and toolset, you could create a new question regarding the implementation of that approach, if you continue to have difficulties.
Related resources
How can I create an executable JAR with dependencies using Maven?
openjfx Runtime images documentation
maven shade plugin
Maven Shade JavaFX runtime components are missing
openjfx JavaFX maven plugin
badass runtime plugin
badass jlink plugin
jlink guide
jpackage script
JEP 392: packaging tool
Warning for shaded jars
If you bundle all JavaFX code into a single jar using the maven shade plugin, you will get a warning like the following when you run your application from Java 16+:
WARNING: Unsupported JavaFX configuration: classes were loaded from 'unnamed module #28c71909'
This indicates that such a configuration is not supported, and may (and probably will) break in future and perhaps current JavaFX platform revisions. Thus, shaded jars that include JavaFX platform code are not recommended by me, even though such jars might currently work for your deployments.
JavaFX 11+ is built to be used as a set of modules. Configurations are not supported if they do not run the JavaFX platform off of the module path but instead run the platform code off of the classpath (as a shaded jar would).

Hive custom UDF class not found issue

I was building my UDF as below:
test_udf.java
package test;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
public class Strip extends UDF{
private Text result = new Text();
public Text evaluate(Text str){
if ( str == null){
return null;
}
result.set(StringUtils.strip(str.toString()));
return result;
}
public Text evaluate(Text str,String stripChars){
if ( str == null){
return null;
}
result.set(StringUtils.strip(str.toString(),stripChars));
return result;
}
}
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>sdf.dennis.com</groupId>
<artifactId>test_udf</artifactId>
<version>1.0</version>
<name>hive</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>3.0.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I used command mvn clean package. Then a test_udf-1.0.jar was generated in target folder.
However, I added the jar file into hive and create a temporary function, it showed:
hive> add jar file:///home/dennis/java/target/test_udf-1.0.jar;
Added [file:///home/dennis/java/target/test_udf-1.0.jar] to class path
Added resources: [file:///home/dennis/java/target/test_udf-1.0.jar]
hive> create temporary function test_s as 'test.strip';
FAILED: Class test.strip not found
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
hive> create temporary function test_s as 'test.Strip';
FAILED: Class test.Strip not found
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
I can't figure out what the mistake I made?
unziped files:
about_files groovyjarjarantlr jodd module.properties plugin.properties
about.html groovyjarjarasm junit mozilla plugin.xml
antlr groovyjarjarcommonscli keytab.txt net properties.dtd
assets hdfs-default.xml krb5-template.conf org PropertyList-1.0.dtd
au hive-exec-log4j2.properties krb5_udp-template.conf org-apache-calcite-jdbc.properties schema
ccache.txt hive-log4j2.properties license org.apache.hadoop.application-classloader.properties shaded
codegen images LICENSE.txt org.codehaus.commons.compiler.properties stylesheet.css
com javaewah log4j2.component.properties overview.html templates
common-version-info.properties javax Log4j-config.xsd overviewj.html testpool.jocl
core-default.xml javolution Log4j-events.dtd package.jdo tez-container-log4j2.properties
fr jersey Log4j-events.xsd parquet webapps
google jetty-dir.css Log4j-levels.xsd parquet-logging.properties yarn-default.xml
groovy jline META-INF parquet.thrift yarn-version-info.properties

May be try with Macros.
CREATE TEMPORARY MACRO fn_maskNull(input decimal(25,3))
CASE
WHEN input IS NULL THEN 0 else input
END;
-- usage
select fn_maskNull(null), fn_maskNull(101);
More info : https://medium.com/#gchandra/create-user-defined-functions-in-hive-beeline-ff965285d735

Maven - Executable file from a java project

I need to use maven (for a school project) to create an executable file from a single maven command. I've never used maven and tried many solutions here on stackoverlow. The solutions created a jar file, but the file never opened.
This is my project structure
src
com
project
code
swing
programm
interface
Main.class
I know this isn't maven convention, however changing it now would mean I would have to adjust the imports (as intelliJ doesn't refactor everything perfectly) for around 40 classes.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>MyGroup</groupId>
<artifactId>myProgramm</artifactId>
<version>0.7-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Hello World</name>
<description>Course Project</description>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc -->
<dependency>
<groupId>org.xerial</groupId>
<artifactId>sqlite-jdbc</artifactId>
<version>3.25.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.junit/junit5-engine -->
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit5-engine</artifactId>
<version>5.0.0-ALPHA</version>
</dependency>
</dependencies>
<build>
</build>
What do I have to put inside to make an executable file?

TimurJD's answer is correct however I would like to explain step by step what is actually happening and why.
To have a jar be executable the JVM needs to know where your main method is.
For that you need a file called META-INF/MANIFEST.MF inside the jar you create.
This file must contain a reference to the class containing your main method which is done like this:
Main-Class: com.example.ClassContainingMainMethod
There are many ways of creating said file but since you are using maven here is the plugin that will help you create this manifest file
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.1</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>com.my.packege.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
Once you have the plugin in your pom.xml simply run the maven install goal, either from your IDE or the command line. After you should find a folder called target in your project. That folder will contain the executable jar.
To run the jar you can call from the command line:
java -jar MyProject.jar
It should also be noted that unless you abide by the maven standard of keeping your source code in src/main/java you will have to specify your source folder explicitly.

You need to add plugin to your pom.xml file
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<configuration>
<mainClass>com.example.Main</mainClass>
</configuration>
</plugin>
</plugins>
</build>
and to run the program: mvn clean install exec:java
... here is the link for doc http://www.mojohaus.org/exec-maven-plugin/usage.html
There are possible different solutions, depends on your requirements: https://www.baeldung.com/executable-jar-with-maven

How to include opencv in an intellij sub module (maven)

I'm using IntelliJ IDE with maven. I have a project (main module) with a parent pom, that includes 2 sub modules, each with their own pom.
<!-- main pom module part -->
<packaging>pom</packaging>
<modules>
<module>ModuleA</module>
<module>ModuleB</module>
</modules>
<!-- example for sub module pom -->
<parent>
<artifactId>main-module</artifactId>
<groupId>my.main.module</groupId>
<version>0.5.0</version>
</parent>
Image ModuleA includes the OpenCV Java wrapper and ModuleB is an executable java program (having the main class) using ModuleA.
The compiling works fine, but when I run ModuleB with having set the library path in the launcher, I'll get the following error for ModuleA:
java.lang.NoClassDefFoundError: org/opencv/core/Core
Any suggestions how to fix this?

Ok, I found a solution my self. The problem was, that the opencv java wrapper was included with a system path. Now I use the maven install plugin within the validate live cycle step instead.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<groupId>org.opencv</groupId>
<artifactId>opencv</artifactId>
<version>3.3.0</version>
<packaging>jar</packaging>
<file>${project.basedir}/../lib/opencv/opencv-330.jar</file>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Works fine for me, but was not the way I wanted it to be... The system-path type dependency seems to be buggy in maven.

Try to add the following dependency to your ModuleA:
<dependency>
<groupId>nu.pattern</groupId>
<artifactId>opencv</artifactId>
<version>2.4.9-7</version>
</dependency>