I'm trying to create a Kafka Streams application with Java and having a hard time managing the dependencies. Please let me say that I am completely noob to Java and it's ecosystem of tools.
The project is compiled with Maven and I'm using IntelliJ Idea. The project is configured to use OpenJDK 14.
Here's the pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.petite</groupId>
<artifactId>ora</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>14</source>
<target>14</target>
</configuration>
</plugin>
</plugins>
<defaultGoal>compile</defaultGoal>
</build>
<dependencies>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
<version>2.5.0</version>
</dependency>
</dependencies>
</project>
With this setup I can print a "Hello world!" to the console but as soon as I import Kafka Streams classes errors start to pop up:
package com.petite;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.kstream.StreamsBuilder;
import org.apache.kafka.streams.processor.Topology;
public class Main {
public static void main(String[] args) {
System.out.println("Hello from Java");
}
}
IntelliJ tells me that it can't find the symbols StreamsBuilder and Topology. Now being a complete alien to Java, IntelliJ and Maven I really can't find a way to go forward.
How come the dependencies shown on Kafka site (https://kafka.apache.org/25/documentation/streams/developer-guide/write-streams.html#libraries-and-maven-artifacts) can be downloaded but can't be imported?
You StreamsBuilder and Topology are under stream package and not in kstream and processor
Use the below imports :
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.Topology;
And as you mentioned that you are new to Java, just an torubleshoot advice , if you ever face such issues just delete the import then modern IDE like Inteliij will automatically show the option.
Related
Writing code on Eclipse with Weka. Trying to use the weka.filters.Filter method but I constantly receive this error:
java.lang.NoClassDefFoundError: weka/filters/Filter
Everything else seems to be imported right and I see the filter class in the Package Exploration.
Thank you to everyone in advance
Haven't used Eclipse in a long time, as I prefer IntelliJ IDEA.
Here is what I did:
Create a new workspace
Create a new Java project
Unchecked Create module-info.java and clicked on Next
Added my external weka.jar under Classpath and clicked on Finish:
Created class Testing.java in package myweka with this content:
package myweka;
import weka.filters.Filter;
import weka.filters.MultiFilter;
public class Testing {
public static void main(String[] args) throws Exception {
Filter f = new MultiFilter();
System.out.println(f);
}
}
Here it is what it looks like in Eclipse:
Executed the class without problems (just outputs the classname in the console).
Instead of simply adding an external jar to your project, you could set up a Maven project. Here are the steps:
Create a new directory to house all your code
Create a file called pom.xml (Maven Project Object Model) and add this content (the libraries that are required for the project are listed inside the dependencies tag, this example uses Weka 3.9.5; the 3.8.5 dependency is commented out; you can only use either or):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>MyGroup</groupId>
<artifactId>MyWeka</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>MyWeka</name>
<description>Project using Weka.</description>
<dependencies>
<!-- Weka 3.9.5 -->
<dependency>
<groupId>nz.ac.waikato.cms.weka</groupId>
<artifactId>weka-dev</artifactId>
<version>3.9.5</version>
</dependency>
<!-- Weka 3.8.5 -->
<!--dependency>
<groupId>nz.ac.waikato.cms.weka</groupId>
<artifactId>weka-stable</artifactId>
<version>3.8.5</version>
</dependency-->
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.3.2</version>
</plugin>
</plugins>
</build>
</project>
Create directory src/main/java/myweka in the same directory as the pom.xml file.
Place the following content as Testing.java in that directory:
package myweka;
import weka.filters.Filter;
import weka.filters.MultiFilter;
public class Testing {
public static void main(String[] args) throws Exception {
Filter f = new MultiFilter();
System.out.println(f);
}
}
Launch Eclipse and create a new workspace.
Import your project
Import the project as an Existing Maven Project
Select the directory where your pom.xml is located and click on Finish.
Once Eclipse has finished the import, you can execute the myweka.Testing class, which will just output the filter's classname in the console.
I am trying to import a library from my maven repo. But I got the error saying "The import org cannot be resolved", screenshot below:
However, we can see the jar under the Maven Dependencies from the Package explorer:
Also, my pom.xml looks like:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Test1</groupId>
<artifactId>Test1</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<release>10</release>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.webjars.npm</groupId>
<artifactId>gregorian-calendar</artifactId>
<version>4.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.8</version>
</dependency>
</dependencies>
</project>
What did I miss here? What else do I need to import org.apache.commons.lang3.time.DateUtils ? Thanks!
Note: I am on Mac
It looks like you have a module-info.java.
In this case, you need to add requires org.apache.commons.lang3; to it (this is the automatic module name from the MANIFEST.MF inside commons-lang3-3.8.jar).
Also, having MainTest1 in the default package is not allowed in a named module, so you have to move it into a package.
Alternatively, you can remove the module-info.java and things should mostly work like before Java 9
I have a very simple application in Maven that uses Apache Felix as an OSGi Framework.
Here is a pom.xml file content:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.installer</groupId>
<artifactId>bundle-installer</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.framework</artifactId>
<version>5.6.10</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<inherited>true</inherited>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
In the main class I am just getting the bundle by its path and
installing it by Apache Felix:
package com.example.installer;
import org.apache.felix.framework.Felix;
import org.osgi.framework.BundleException;
import java.util.Properties;
public class Main {
public static void main(String[] args) throws BundleException {
Properties config = new Properties();
config.setProperty("org.osgi.framework.storage", "/Users/johndoe/plugins");
Felix framework = new Felix(config);
framework.start();
framework.getBundleContext().installBundle("file:/Users/johndoe/Desktop/bundle-example-0.0.0.1.jar");
framework.stop();
}
}
Let me explain what I want to implement. As you see I have changed Felix cache dir to "/Users/johndoe/plugins", i.e. I am going to install plugins programmatically by Felix. This code is working well enough except one thing. After installation each bundle is being installed in the folder like bundle0, bundle1, ..., bundleN etc. But I don't like this, I want each bundle to be installed in the folder according Symbolic name like: xmlreader, code-analyzer etc.
Unfortunately it is not possible by setting it in Felix config properties. But I hope that there should be some ready solutions instead inventing the wheel.
How to implement this feature in proper way?
I am working on a Hello World application. My application is built with Maven (3.5.0), uses Apache Felix annotations, and is run in Apache Karaf (4.1.1). My application consists of a single component, called App, that should be started immediately. The bundle builds successfully. I can successfully install it into Karaf from my mvn repository. Karaf shows the bundle as "Active". The problem is that constructor and the activate method of my component (App) are never invoked. I need a second set of eyes to help me figure out why this happens. What am I missing in my pom.xml?
For completeness, I created an Activator class in my project that implements BundleActivator. I then instructed Maven to set my Bundle-Activator to this new class. Now, when I install my bundle in Karaf, I can see the log output from my Activator. The start method is being hit. So I know my bundle is actually starting. I just don't understand why my App component is never created and activated.
Here are the relevant files.
App.java
package myCompany;
import org.apache.felix.scr.annotations.Activate;
import org.apache.felix.scr.annotations.Component;
#Component(immediate=true)
public class App
{
public App()
{
System.out.println( "App constructed" );
}
#Activate
public void activate()
{
System.out.println( "App activated" );
}
}
Activator.java
package myCompany;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
public class Activator implements BundleActivator{
public void start(BundleContext context) throws Exception {
System.out.println("Activator started");
}
public void stop(BundleContext context) throws Exception {
System.out.println("Activator stopped");
}
}
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>myCompany</groupId>
<artifactId>myProject</artifactId>
<packaging>bundle</packaging>
<version>1.0-SNAPSHOT</version>
<name>myProject</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.core</artifactId>
<version>6.0.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.scr.annotations</artifactId>
<version>1.9.6</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-scr-plugin</artifactId>
<version>1.24.0</version>
<executions>
<execution>
<id>generate-scr-scrdescriptor</id>
<goals>
<goal>scr</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<version>3.3.0</version>
<configuration>
<instructions>
<_dsannotations>*</_dsannotations>
<_metatypeannotations>*</_metatypeannotations>
<Bundle-Activator>myCompany.Activator</Bundle-Activator>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
</project>
Karaf does not include support for declarative services out of the box. You need to install the scr feature to activate DS support:
feature:install scr
You are using obsolete annotations from the org.apache.felix.scr.annotations package, which are not recognised or processed by bnd — they require an additional Maven plugin to work.
It would be better to migrate your code to use the OSGi standard annotations from the org.osgi.service.component.annotations package. See OSGi Compendium Release 6 specification, section 112.8.
I have a problem getting BioJava to work in a Netbeans RCP application, built using Maven. I've created a Maven module as a wrapper, including org.biojava.* and org.forester.* packages as public in the POM. Then, from another module I set the wrapper as a dependency, and use some of the basic examples from the BioJava cookbook for testing.
Whenever I try to instantiate some object of a class from BioJava, the application freezes and I have to kill it using the Windows task manager.
Here's the wrapper's pom file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>nl.hecklab.bioinformatics</groupId>
<artifactId>Spider-parent</artifactId>
<version>1.0.0</version>
</parent>
<artifactId>BiojavaWrapper</artifactId>
<version>4.1.0</version>
<packaging>nbm</packaging>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>nbm-maven-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<useOSGiDependencies>true</useOSGiDependencies>
<publicPackages>
<publicPackage>org.biojava.*</publicPackage>
<publicPackage>org.forester.*</publicPackage>
</publicPackages>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<useDefaultManifestFile>true</useDefaultManifestFile>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.biojava</groupId>
<artifactId>biojava-alignment</artifactId>
<version>4.1.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.12</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
</project>
Here's some code I try to get to work. This is just a very coarse example, called from a button in a TopComponent. Input and output are just text fields.
private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {
Reader r = new Reader(new File("D:\\current\\fastafile.fasta"));
for (ProteinSequence a : r.getSequences()) {
input.append(a.toString());
}
Profile<ProteinSequence, AminoAcidCompound> profile = Alignments.getMultipleSequenceAlignment(r.sequences);
output.setText(String.format("Clustalw:%n%s%n", profile));
ConcurrencyTools.shutdown();
}
Here's the reader class:
public class Reader {
List<ProteinSequence> sequences = new ArrayList<>();
public Reader(File fastaFile) {
try {
FileInputStream inStream = new FileInputStream(fastaFile);
FastaReader<ProteinSequence, AminoAcidCompound> fastaReader
= new FastaReader<>(
inStream,
new GenericFastaHeaderParser<ProteinSequence, AminoAcidCompound>(),
new ProteinSequenceCreator(AminoAcidCompoundSet.getAminoAcidCompoundSet()));
LinkedHashMap<String, ProteinSequence> b = fastaReader.process();
sequences.addAll(b.values());
} catch (IOException ex) {
Logger.getLogger(Reader.class.getName()).log(Level.SEVERE, null, ex);
}
}
public List<ProteinSequence> getSequences() {
return sequences;
}
}
In the (Netbeans) IDE, the classes are found and used in autocompletion, and the project builds successfully, in each case indicating that principally the dependencies are set up correctly.
First of all check the wrapper module's manifest to see if all entries are correctly generated, especially since you define useOSGiDependencies==true. It could be that the biojava jars contain osgi headers and then you are not wrapping the jars in module, but declare a dependency on osgi plugin.
However locking of the app is weird, if there was something wrong with the runtime dependencies I would have expected an early 'unsatisfied dependencies' error. You might want to create a thread dump and check what's going on. Maybe you have a deadlock. Or since your action (jButton1ActionPerformed) is called from AWT, maybe the whole reading thing just takes time and your UI thread is locked.
I've done a lot of searching and found that the actual culprit is slf4j, that's used throughout BioJava.
I don't know why it freezes the platform application, but I'm able to cause my module to not install by creating a slf4j logger in it.
I've seen a solution online for a wrapper module, and it turns out it's enough to create a wrapper for org.slf4j:slf4j-api:x.y.z together with org.slf4j:slf4j-jdk14:x.y.z. Add org.slf4j.* to the public packages. Here's the POM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>group</groupId>
<artifactId>parent-project</artifactId>
<version>1.0.0</version>
</parent>
<artifactId>slf4jwrapper</artifactId>
<packaging>nbm</packaging>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>nbm-maven-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<publicPackages>
<publicPackage>org.slf4j.*</publicPackage>
</publicPackages>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<useDefaultManifestFile>true</useDefaultManifestFile>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.netbeans.api</groupId>
<artifactId>org-netbeans-api-annotations-common</artifactId>
<version>${netbeans.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>1.7.7</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
The wrapper should then be used in the BioJava dependent modules, but it should work for other modules depending on slf4j as well.