org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException - java

I don't know what it wants from me. I am using
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>${deeplearning4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-nlp</artifactId>
<version>${deeplearning4j.version}</version>
</dependency>
where
<deeplearning4j.version>0.4-rc3.8</deeplearning4j.version>
but I am getting
Caused by: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: null
at org.nd4j.linalg.factory.Nd4jBackend.load(Nd4jBackend.java:148) ~[nd4j-api-0.4-rc3.7.jar:na]
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:4498) ~[nd4j-api-0.4-rc3.7.jar:na]
... 53 common frames omitted
if I try to load the Google word vector model:
#RequestMapping("/loadModel")
public Boolean loadModel(#RequestParam(value="model") String model) {
Resource resource = appContext.getResource("WEB-INF/word-vector-models/" + model);
try {
File modelFile = resource.getFile();
System.err.println(modelFile.getAbsolutePath());
WordVectors googleModel = WordVectorSerializer.loadGoogleModel(modelFile, true);
this.wordVectorsMap.put(model, googleModel);
} catch (IOException e) {
e.printStackTrace();
return false;
}
return true;
}


It looks like you don't have an nd4j backend specified in your pom file. You have to have one, and you should use only one (don't have multiple backends in your pom at once unless you use profiles). Currently, for version 0.4-rc3.8, I've had luck with nd4j-x86 on non-GPU enabled Mac, Windows and linux boxes. If you have access to GPUs, you could use one of the nd4j-jcublas-7.x jars, but be aware that there is a major Cuda rewrite going on according to their Gitter.
For now,
Read the "Dependencies and Backends" section of http://deeplearning4j.org/quickstart.html,
Determine which backend is right for you at http://nd4j.org/dependencies.html (Note that Jblas is only available if you downgrade to version 0.4-rc3.6)
Keep an eye on the dl4j Gitter
If you're trying to use GPUs like I am, keep an eye on Github issue #555
Here's how I set up my pom.xml dependencies. By default ((i.e. mvn clean install), it runs with nd4j-x86, but when I pull my code onto the GPU box, I just append the profile name (so mvn clean install -P cuda) and switch backends easily:
<!-- Platform-dependent backend selection (netlib is default) -->
<profiles>
<profile>
<id>cuda</id>
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-jcublas-${cuda.version}</artifactId>
<version>${nd4j.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>netlib</id>
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-x86</artifactId>
<version>${nd4j.version}</version>
</dependency>
</dependencies>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
</profiles>
<!-- end platform-dependent backend selection -->
<dependencies>
<!-- dl4j dependencies -->
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-ui</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-scaleout-api</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-scaleout-akka</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-scaleout-zookeeper</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-nlp</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-aws</artifactId>
<version>${dl4j.version}</version>
</dependency>
<!-- end dl4j dependencies -->
<!-- nd4j dependencies -->
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>canova-nd4j-image</artifactId>
<version>${canova.version}</version>
</dependency>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>canova-nd4j-codec</artifactId>
<version>${canova.version}</version>
</dependency>
<!-- end nd4j dependencies -->
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>net.java.openjfx.backport</groupId>
<artifactId>openjfx-78-backport</artifactId>
<version>1.8.0-ea-b96.1</version>
</dependency>
<!-- logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.13</version>
</dependency>
<!-- end logging -->
<dependency>
<groupId>org.apache.maven.reporting</groupId>
<artifactId>maven-reporting-api</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>

Related

Classes loaded in classloader are note the same when launching in Intellij and Maven command on JUnit test

I have a git module project using maven and wicket and imported in IntelliJ.
I have also in a module, one test trying to test to load all classes present in dependencies used by the project to check if there is no encoding error in file *.properties (and other tasks).
When i launch it directly from Intellij(green play button close to the test name), the test is working and all classes from all dependencies are loaded, but when i launch with
mvn clean install
in terminal or from maven toolbar only classes in current project are loaded.
i tried all classloader possibilities but each time same result, with maven command only class in target/classes of current project are loaded in Classloader
this is my test
#Test
void testCheckMistakesInProperties() throws Exception {
var iter = Reflections.class.getClassLoader().getResources( "" ).asIterator();
//var iter = ClassLoader.getPlatformClassLoader().getResources( "" ).asIterator();
//var iter2 = ClassLoader.getSystemClassLoader().getResources( "" ).asIterator();
//var iter3 = AllWicketPagesTest.class.getClassLoader().getResources( "" ).asIterator();
while(iter.hasNext()){
var url = iter.next();
Path resourcesPath;;
try{
resourcesPath = Path.of( url.toURI() );
}catch(FileSystemNotFoundException e){
continue;
}
try( var pathStream = java.nio.file.Files.walk( resourcesPath ) ){
pathStream.forEach( path -> {
var pathStr = path.toString();
LOGGER.info( "Properties file checked {}", pathStr ); });
}catch(Exception e){
continue;
}
}
}
i tried all method (classLoader Helper etc...) nothing working
here my pom.xml
<dependencies>
<dependency>
<groupId>es.lib</groupId>
<artifactId>webapp</artifactId>
</dependency>
<dependency>
<groupId>es.java</groupId>
<artifactId>entities</artifactId>
</dependency>
<dependency>
<groupId>es.java</groupId>
<artifactId>entities</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.lib</groupId>
<artifactId>stockmarket</artifactId>
</dependency>
<dependency>
<groupId>com.java.intranet</groupId>
<artifactId>intranet-web</artifactId>
</dependency>
<dependency>
<groupId>org.wicketstuff</groupId>
<artifactId>wicketstuff-html5</artifactId>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.webjars.npm</groupId>
<artifactId>bootstrap-tourist</artifactId>
</dependency>
<dependency>
<groupId>com.java</groupId>
<artifactId>base-entities</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.java</groupId>
<artifactId>web</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.test-framework</groupId>
<artifactId>jersey-test-framework-core</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.test-framework.providers</groupId>
<artifactId>jersey-test-framework-provider-inmemory</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.icegreen</groupId>
<artifactId>greenmail-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency><!-- To provide the annotation processor in the IDE's classpath -->
<groupId>org.hibernate</groupId>
<artifactId>hibernate-jpamodelgen</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>Clean test database before the tests</id>
<phase>process-test-resources</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
It is a complicated project with a lot of parent pom, but i think only in this project we can fix the issue i want to be able to see in classloader classes from es.java lib or com.java
I think also with intellij it uses the project structure link to load dependencies classes(present as a submodule in editor)
even if i use a class from lib and retrieve classloader it will load only the current project (where my test is launched) to load only target/classes classes and not form jar or external lib/dependencies

Failed to create Blob container in Microsoft Azure

I followed the tutorial of Azure Blob storage service here and supplied the connection string from my storage account to create a container but it did not work as the container is not created and the application runs until I terminate it manually. I used azure-blob-storage API version 12.4.0.
Here is the code:
// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(connectStr).buildClient();
//Create a unique name for the container
String containerName = "quickstartblobs" + java.util.UUID.randomUUID();
// Create the container and return a container client object
BlobContainerClient containerClient = blobServiceClient.createBlobContainer(containerName);
Here is my pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>aaaa</groupId>
<artifactId>ddada</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>11</source>
<target>11</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.327</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage -->
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-storage</artifactId>
<version>1.108.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
<groupId>de.hpi.cloudraid</groupId>
<artifactId>erasure</artifactId>
<version>1.0.3</version>
</dependency>
<dependency>
<groupId>com.hiramsoft.commons</groupId>
<artifactId>jsalparser</artifactId>
<version>0.2-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
<version>1.7</version>
</dependency>
<dependency>
<groupId>com.google.googlejavaformat</groupId>
<artifactId>google-java-format</artifactId>
<version>1.7</version>
</dependency>
<dependency>
<groupId>com.google.crypto.tink</groupId>
<artifactId>tink</artifactId>
<version>1.3.0-rc3</version>
</dependency>
<dependency>
<groupId>dev.morphia.morphia</groupId>
<artifactId>core</artifactId>
<version>1.5.8</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.10</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>2.11.0.rc1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.11.0.rc1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.opencsv/opencsv -->
<dependency>
<groupId>com.opencsv</groupId>
<artifactId>opencsv</artifactId>
<version>5.1</version>
</dependency>
<dependency>
<groupId>com.hierynomus</groupId>
<artifactId>sshj</artifactId>
<version>0.27.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.azure/azure-storage-blob -->
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-blob</artifactId>
<version>12.4.0</version>
</dependency>
</dependencies>
</project>
It also showed another error message like this:
May 29, 2020 3:39:49 PM io.netty.channel.AbstractChannelHandlerContext invokeExceptionCaught
WARNING: An exception 'java.lang.NoSuchMethodError: 'java.lang.String io.netty.handler.codec.http.HttpUtil.formatHostnameForHttp(java.net.InetSocketAddress)'' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:
java.lang.NoSuchMethodError: 'java.lang.String io.netty.handler.codec.http.HttpUtil.formatHostnameForHttp(java.net.InetSocketAddress)'
at reactor.netty.http.client.HttpClientConnect$HttpClientHandler.resolveHostHeaderValue(HttpClientConnect.java:593)
at reactor.netty.http.client.HttpClientConnect$HttpClientHandler.requestWithBody(HttpClientConnect.java:552)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.lambda$onStateChange$0(HttpClientConnect.java:429)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:430)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:473)
at reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.onStateChange(PooledConnectionProvider.java:525)
at reactor.netty.resources.PooledConnectionProvider$PooledConnection.onStateChange(PooledConnectionProvider.java:434)
at reactor.netty.channel.ChannelOperationsHandler.channelActive(ChannelOperationsHandler.java:62)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelActive(CombinedChannelDuplexHandler.java:414)
at io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64)
at io.netty.channel.CombinedChannelDuplexHandler.channelActive(CombinedChannelDuplexHandler.java:213)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:213)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:199)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:192)
at reactor.netty.tcp.SslProvider$SslReadHandler.userEventTriggered(SslProvider.java:731)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:329)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:315)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:307)
at io.netty.handler.ssl.SslHandler.setHandshakeSuccess(SslHandler.java:1480)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1318)
at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1170)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1195)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at java.base/java.lang.Thread.run(Thread.java:830)
Any reasons why it did not work?

Update:
Thanks to Ihsan Haikal for sharing. If anyone who faces similar problems cannot create a container using the following method, please check the content of pom.xml The problem should not be the code but the configuration part.
Original Answer:
With this steps I can create container on my storage:
First, import:
import com.azure.storage.blob.BlobContainerClient;
import com.azure.storage.blob.BlobServiceClient;
import com.azure.storage.blob.BlobServiceClientBuilder;
And this is the code in my pom.xml:
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-blob</artifactId>
<version>12.0.0</version>
</dependency>
Second, use code:
// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString("DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net").buildClient();
//Create a unique name for the container
String containerName = "quickstartblobs";
// Create the container and return a container client object
BlobContainerClient containerClient = blobServiceClient.createBlobContainer(containerName);
You should get connection string from this place:
Then, it creates the container:
You can check what different between you and me.:)

Exception thrown while running unit tests for maven plugin - ComponentLookupException :: NoSuchElementException :: RepositorySystem

I'm trying to run unit tests to a Maven Plugin's Mojo using the artifact maven-plugin-testing-harness:3.3.0, with the following dependencies. But when I try to 'lookupEmptyMojo', the exception below is thrown. Any quick way to solve this configuration issue?
...
<properties>
<maven.api.version>3.6.2</maven.api.version>
<dependencies>
<!-- Maven plugin deps -->
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>${maven.api.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-core</artifactId>
<version>${maven.api.version}</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Tests -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven.plugin-testing</groupId>
<artifactId>maven-plugin-testing-harness</artifactId>
<version>3.3.0</version>
<scope>test</scope>
</dependency>
</dependencies>
...
Exception:
org.codehaus.plexus.component.repository.exception.ComponentLookupException: java.util.NoSuchElementException
role: org.apache.maven.repository.RepositorySystem
roleHint:
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:267)
...
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)
Caused by: java.util.NoSuchElementException
at org.eclipse.sisu.inject.LocatedBeans$Itr.next(LocatedBeans.java:141)
... 28 more

Coming back to report how to had fixed this issue.
I had to add two others dependencies: maven-compat and maven-resolver-api.
Test-harness uses Maven2 classes, so if we are using Maven3 we need to add the lib maven-compat with the same version of maven api that we are using; this lib implements back compatibility with the old api.
RepositorySystem is an interface present in the maven-resolver-api lib, which works with artifact repositories, that can be remote, local, or even build reactor or IDE workspace.
The dependencies were added to the pom like the following:
<dependency>
<groupId>org.apache.maven.resolver</groupId>
<artifactId>maven-resolver-api</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-compat</artifactId>
<version>${maven.api.version}</version>
<scope>test</scope>
</dependency>

Using confluent avro serializer in Java - NoSuchFieldError: PASSWORD

I'm using confluent avro producer in java but while running it getting below issue.I have added all the required jar .Don't have any clue what did i miss?
Exception in thread "main" java.lang.NoSuchFieldError: PASSWORD
at io.confluent.kafka.serializers.AbstractKafkaAvroSerDeConfig.baseConfigDef(AbstractKafkaAvroSerDeConfig.java:92)
at io.confluent.kafka.serializers.KafkaAvroSerializerConfig.<clinit>(KafkaAvroSerializerConfig.java:28)
at io.confluent.kafka.serializers.KafkaAvroSerializer.configure(KafkaAvroSerializer.java:48)
at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.configure(ExtendedSerializer.java:60)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:341)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:302)
I have tried with below configurations.
Properties prop = new Properties();
prop.put("bootstrap.servers", "localhost:9092");
prop.put("schema.registry.url", "http://localhost:8081");
//prop.put("basic.auth.user.info", "");
prop.put("ssl.keystore.type",null);
prop.put("acks", "all");
prop.put("retries", "0");
prop.put("key.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");
prop.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");
prop.put("group.id", "test");
These are the dependency which i used it.All the jars with latest version.
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-registry</artifactId>
<version>4.1.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent/common-config -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>common-config</artifactId>
<version>3.3.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent/common-utils -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>common-utils</artifactId>
<version>3.3.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent/kafka-connect-avro-converter -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-connect-avro-converter</artifactId>
<version>5.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent/kafka-schema-registry-client -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-registry-client</artifactId>
<version>5.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent.kafka/connect-utils -->
<dependency>
<groupId>io.confluent.kafka</groupId>
<artifactId>connect-utils</artifactId>
<version>0.1.18</version>
</dependency>
<dependency>
<groupId>io.confluent.kafka</groupId>
<artifactId>connect-checkstyle</artifactId>
<version>0.1.18</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.confluent/rest-utils -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>rest-utils</artifactId>
<version>4.1.0</version>
</dependency>

I've seen this error when mixing versions.
You should only need this as the common util, client, rest-utils, etc, are all transitive dependencies
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>5.2.1</version>
</dependency>
https://docs.confluent.io/current/app-development/index.html#native-clients-with-serializers
Remove the rest of the dependencies, or at least create a property for the version and re-use it

log4j.properties being ignored

I have a Java EE 6 Wicket application deployed with maven using IntelliJ IDEA 9.0.3 on glassfish v3.0.1. I use slf4j-log4j12-1.5.6 with slf4j-api-1.5.8 and log4j-1.2.16 for logging.
It was previously working fine when I deployed through netbeans or eclipse, however when I deploy with IntelliJ IDEA my log4j.properties file is ignored and glassfish's logging handles my log messages. I do not think IDEA has anything to do with it, something else must have changed I just can't figure out what.
I have verified that my log4j.properties file is in my WEB-INF/classes directory and the slf4j/log4j jars are in the WEB-INF/lib directory of my war. Is there some sort of configuration I am missing to make this work?
thanks.
edit: Updated with more info, posted pom dependencies.
Here is the relevant section from my pom.xml:
<!-- Guava -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>r05</version>
</dependency>
<!-- Test -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.glassfish.extras</groupId>
<artifactId>glassfish-embedded-all</artifactId>
<version>3.0</version>
<scope>test</scope>
</dependency>
<!-- Java EE 6 -->
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>bean-validator</artifactId>
<version>3.0-JBoss-4.0.0.Beta3</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
<!-- Wicket -->
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket</artifactId>
<version>1.4.9</version>
</dependency>
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-auth-roles</artifactId>
<version>1.4.9</version>
</dependency>
<dependency>
<groupId>org.jboss.weld</groupId>
<artifactId>weld-wicket</artifactId>
<version>1.0.1-Final</version>
</dependency>
<!-- Hibernate -->
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.5.1-Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-annotations</artifactId>
<version>3.5.1-Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-commons-annotations</artifactId>
<version>3.2.0.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>3.5.1-Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-c3p0</artifactId>
<version>3.5.1-Final</version>
</dependency>
<!-- Database -->
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>8.4-701.jdbc4</version>
</dependency>
<!-- Logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.5.6</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
</dependencies>

Update: I tried to reproduce the issue. I created a simple Wicket project (same version as you):
mvn archetype:create \
-DarchetypeGroupId=org.apache.wicket \
-DarchetypeArtifactId=wicket-archetype-quickstart \
-DarchetypeVersion=1.4.9 \
-DgroupId=com.mycompany \
-DartifactId=my-wicketapp
Which has a simple log4j.properties logging to the standard output.
log4j.appender.Stdout=org.apache.log4j.ConsoleAppender
log4j.appender.Stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.Stdout.layout.conversionPattern=%-5p - %-26.26c{1} - %m\n
log4j.rootLogger=INFO,Stdout
log4j.logger.org.apache.wicket=INFO
log4j.logger.org.apache.wicket.protocol.http.HttpSessionStore=INFO
log4j.logger.org.apache.wicket.version=INFO
log4j.logger.org.apache.wicket.RequestCycle=INFO
Then:
I added all your dependencies (or modified the versions of existing one to match yours)
I just did some cleanup e.g. in the Hibernate dependencies, you don't need to declare them all, leverage the transitive dependencies mechanism
I added relevant repositories and pluginRepositories
I added glassfish's javax.servlet dependency to make the build pass
I added the embedded-glassfish plugin to test the whole thing
I made a few other unrelated changes
I changed the compiler settings to 1.6
I declared slf4j-api in the dependencyManagement element to control nicely the version in transitive dependencies.
The full pom.xml looks like this (so anybody can reproduce):
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany</groupId>
<artifactId>my-wicketapp</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<!-- TODO project name -->
<name>quickstart</name>
<description/>
<!--
TODO <organization> <name>company name</name> <url>company url</url>
</organization>
-->
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<repositories>
<!-- For Hibernate Artifacts -->
<repository>
<id>repository.jboss.org-public</id>
<name>JBoss repository</name>
<url>https://repository.jboss.org/nexus/content/groups/public</url>
</repository>
<!-- repository for Java EE 6 Binaries -->
<repository>
<id>java.net2</id>
<name>Repository hosting the jee6 artifacts</name>
<url>http://download.java.net/maven/2</url>
</repository>
</repositories>
<pluginRepositories>
<!-- GlassFish repository for the embedded-glassfish plugin -->
<pluginRepository>
<id>glassfish</id>
<name>GlassFish Maven 2 Repository</name>
<url>http://download.java.net/maven/glassfish</url>
</pluginRepository>
</pluginRepositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>3.5.5-Final</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>r05</version>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>javax.servlet</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>bean-validator</artifactId>
<version>3.0-JBoss-4.0.0.Beta3</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>6.0</version>
<scope>provided</scope>
</dependency>
<!-- WICKET DEPENDENCIES -->
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket</artifactId>
<version>${wicket.version}</version>
</dependency>
<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-auth-roles</artifactId>
<version>${wicket.version}</version>
</dependency>
<dependency>
<groupId>org.jboss.weld</groupId>
<artifactId>weld-wicket</artifactId>
<version>1.0.1-Final</version>
</dependency>
<!--
OPTIONAL <dependency> <groupId>org.apache.wicket</groupId>
<artifactId>wicket-extensions</artifactId>
<version>${wicket.version}</version> </dependency>
-->
<!-- LOGGING DEPENDENCIES - LOG4J -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<!-- JUNIT DEPENDENCY FOR TESTING -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
<!-- GLASSFISH EMBEDDED FOR TESTING -->
<dependency>
<groupId>org.glassfish.extras</groupId>
<artifactId>glassfish-embedded-all</artifactId>
<version>3.0.1</version>
<scope>test</scope>
</dependency>
<!-- JETTY DEPENDENCIES FOR TESTING -->
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<version>${jetty.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<version>${jetty.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-management</artifactId>
<version>${jetty.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<resources>
<resource>
<filtering>false</filtering>
<directory>src/main/resources</directory>
</resource>
<resource>
<filtering>false</filtering>
<directory>src/main/java</directory>
<includes>
<include>**</include>
</includes>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</resource>
</resources>
<testResources>
<testResource>
<filtering>false</filtering>
<directory>src/test/java</directory>
<includes>
<include>**</include>
</includes>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</testResource>
</testResources>
<plugins>
<plugin>
<inherited>true</inherited>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
<optimize>true</optimize>
<debug>true</debug>
</configuration>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<downloadSources>true</downloadSources>
</configuration>
</plugin>
<plugin>
<groupId>org.glassfish</groupId>
<artifactId>maven-embedded-glassfish-plugin</artifactId>
<version>3.0.1</version>
<configuration>
<serverID>server</serverID>
<name>server</name>
<app>${project.build.directory}/${project.build.finalName}.war</app>
<port>8080</port>
<instanceRoot>${project.build.directory}/gfe-${maven.build.timestamp}</instanceRoot>
<!--contextRoot>${build.finalName}</contextRoot-->
<autoDelete>true</autoDelete>
<!--configFile>${basedir}/domain.xml</configFile-->
</configuration>
</plugin>
</plugins>
</build>
<properties>
<wicket.version>1.4.9</wicket.version>
<jetty.version>6.1.4</jetty.version>
<slf4j.version>1.5.6</slf4j.version>
</properties>
</project>
And when I run the project with the embedded-glassfish plugin:
$ mvn package
...
$ mvn embedded-glassfish:run
...
and access http://localhost:8080/server in a browser, I get my logs in the standard output as expected:
...
INFO: [WicketApplication] Started Wicket version 1.4.9 in development mode
********************************************************************
*** WARNING: Wicket is running in DEVELOPMENT mode. ***
*** ^^^^^^^^^^^ ***
*** Do NOT deploy to your live server(s) without changing this. ***
*** See Application#getConfigurationType() for more information. ***
********************************************************************
I wonder if this is representative or not.
I have checked the war, log4j.properties is indeed in WEB-INF/classes. I don't have a log4j.jar, i have slf4j-log4j12.jar.
slf4j-log4j12.jar is not a replacement for log4j.jar, slf4j-log4j12.jar is a binding for log4J version 1.2, you still need log4j.jar. From the SLF4J documentation:
Binding with a logging framework at deployment time
As mentioned previously, SLF4J
supports various logging frameworks.
The SLF4J distribution ships with
several jar files referred to as
"SLF4J bindings", with each binding
corresponding to a supported
framework.
slf4j-log4j12-1.6.1.jar: Binding for
log4j version 1.2, a widely used
logging framework. You also need to
place log4j.jar on your class path.
I wonder how you got this working under NetBeans and Eclipse.

I had the exact same problem.
Log4j alone works great:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
But if I try to use Slf4j over it, then my "src/resources/log4j.properties" file is not found anymore, even if Maven add it to a directory that is on the classpath.
So, this doesn't work out of the box:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.4</version>
</dependency>
To make it work, you have to explicitly add the "log4j.properties" to the classpath or tell the server where to find it! A way to achieve this is (this example is on Windows):
-Dlog4j.configuration=file:C:\[pathToYourProject]\trunk\target\classes\log4j.properties
In Eclipse (if this is what you use), you can add the same line to your Run Configuration / VM arguments.

I suggest removing all the slf4j dependencies and change your logging code to use the Log4j API directly (if this is not too much work, don't know the size of your project). Once this works, consider if you really need the flexibility offered by slf4j. If you do, pick up the correct version of slf4j (slf4j-log4j12 might not be the correct one to use for log4j 1.2.16) and integrate it back in.
I encountered slf4j recently, and in the end removed it altogether because I ran into similar configuration problems.

I had the same problems. The solution is simple - all logging dependencies should be before the glassfish on the classpath.
Note that older Maven2 versions have some problems with the classpath consistency. I use 2.2.1 which have this issue fixed (it was fixed in 2.0.9, I think).

Have a look at the log4j manual. The section "Default Initialization Procedure" describes how log4j will try to find the initialization file. Maybe you can try some of this options to get things work.

The two most likely things that spring to mind are:
make sure the log4j.properties file is in WEB-INF/classes of the deployment. I assume you mean it is in your WEB-INF/classes in your codebase, but have you confirmed this is the case in the war that sent to glassfish
make sure your log4j.jar is in your deployed WEB-INF/lib
Since it's working in some deployment scenarios, I suspect your war packaging when running via Maven is the problem. The above points should help you confirm this.

I have the following logging dependencies :
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>0.9.17</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.5.8</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.5.8</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
<scope>runtime</scope>
</dependency>
under /src/main/resources/ i have a logback.xml defining the various aspects (appenders,..). This gets picked up by maven and copied to WEB-INF/classes
hope that helped

I would support Adriaan Koster's answer. If your pure log4j doesn't work however, try the following. Create simple class like this
import org.apache.log4j.Logger;
public class LogTest {
private static Logger log;
public static void main(String[] args) {
log = Logger.getLogger(LogTest.class);
}
}
...and put breakpoint in org.apache.log4j.helpers.Loader#getResource method. It tries to pull log4j.xml from classloader:
url = classLoader.getResource(resource);
So you can easily look inside the classloader and see what paths it uses (and why it can't find your log4j.xml).

It is also very important that log4j library was compiled after all modules of the libraries used in the project. If you will not set it as last object, the logs from later modules/libraries wouldn't be displayed in a standard way for log4j.properties.

For people using spring boot:
It appears to ignore -Dlog4j.configuration=file:... You can instead use -Dlogging.config=file:/etc/myconfig/log4j.xml. It must be xml format.
You can also edit the file logback-spring.xml in your java source dir eg src/main/resources/. This is the standard place for the default logging config.
A good source of further information: https://www.baeldung.com/spring-boot-logging

What worked for me: Adding log4j2.properties (or log4j2.xml) to my class path root (tomcat/webapps/ROOT/WEB-INF/classes/). Parameter-value logging is now enabled.
Background: I noticed log4j 1, 2 and slf4j jars in my build. We have slf4j imports in many java files. hibernate.properties was being used and hibernate.show_sql=true worked but I could not enable parameter-value logging. My log4j.properties in the classpath root was being ignored - when the mvn packaged war was run in tomcat. But when I ran it from eclipse, log4j.properties worked and I could enable parameter-value logging.
log4j2.properties used:
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
rootLogger.level = info
rootLogger.appenderRef.stdout.ref = STDOUT
logger.hibernate.name=org.hibernate.SQL
logger.hibernate.level=debug
logger.hibernate-type.name=org.hibernate.type
logger.hibernate-type.level=trace
Based on: https://www.codejava.net/frameworks/hibernate/how-to-configure-hibernate-logging-with-log4j2
Search for log4j/slf4j: find /usr/local/tomcat/webapps/ROOT -name '*slf4j*' -o -name '*log4j*'

Categories

Resources