Datadog instrumentation on Spring boot container application - class not found - java

I'm currently running a spring boot application as container into a kubernetes cluster.
Datadog agent is running as containers on the cluster.
I have modified the container image build to include the datadog agent before running the application:
CMD ["/bin/sh","-c","java -javaagent:dd-java-agent.jar -Ddd.trace.config=datadog.properties -Dlogging.config=/logback.xml -jar service.jar"]
I also setup the environment variable to indicate the HOST IP of the agent to my container via the Deployment file.
The problem now is i'm getting this class not found exception when the application starts:
java.lang.NoClassDefFoundError: datadog/trace/instrumentation/springscheduling/SpringSchedulingInstrumentation
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at org.springframework.core.annotation.AnnotationsScanner.processClassHierarchy(AnnotationsScanner.java:233)
at org.springframework.core.annotation.AnnotationsScanner.processClassHierarchy(AnnotationsScanner.java:194)
at org.springframework.core.annotation.AnnotationsScanner.processClass(AnnotationsScanner.java:128)
at org.springframework.core.annotation.AnnotationsScanner.process(AnnotationsScanner.java:107)
at org.springframework.core.annotation.AnnotationsScanner.scan(AnnotationsScanner.java:97)
at org.springframework.core.annotation.AnnotationsScanner.scan(AnnotationsScanner.java:78)
at org.springframework.core.annotation.TypeMappedAnnotations.scan(TypeMappedAnnotations.java:242)
at org.springframework.core.annotation.TypeMappedAnnotations.get(TypeMappedAnnotations.java:149)
at org.springframework.core.annotation.TypeMappedAnnotations.get(TypeMappedAnnotations.java:131)
at org.springframework.core.annotation.AnnotationUtils.findAnnotation(AnnotationUtils.java:581)
at org.springframework.cloud.sleuth.DefaultSpanNamer.annotation(DefaultSpanNamer.java:71)
at org.springframework.cloud.sleuth.DefaultSpanNamer.name(DefaultSpanNamer.java:58)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.<init>(TraceRunnable.java:59)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.<init>(TraceRunnable.java:51)
at org.springframework.cloud.sleuth.instrument.async.LazyTraceThreadPoolTaskScheduler.scheduleWithFixedDelay(LazyTraceThreadPoolTaskScheduler.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.cloud.sleuth.instrument.async.ExecutorMethodInterceptor.invoke(ExecutorBeanPostProcessor.java:328)
Quite straigtforward, i need to include some dependencies into the application package. But i could not find anything useful on datadog website nor maven central repository.
Including the agent itself or the api libraries fix nothing. This class is present on the agent but under a different path.
Does anybody know which dependencies should be included in the classpath of the application to fix that ?

Spring is scanning your classpath that seems incomplete. Maybe it is related to Spring's class loading mechanisms. The class in question exists and seems to be part of the agent.
Possibly, you are using an outdated version of the agent.

Related

Java jetty servlet container fails to resolve class dependencies

15:28:38.716 [qtp1588771273-32] WARN o.e.jetty.servlet.ServletHandler - /tp/gremlin/execute
java.lang.IllegalArgumentException: Could not resolve dependency of type:javax.transaction.TransactionManager
at org.neo4j.graphdb.DependencyResolver$Adapter$1.select(DependencyResolver.java:87) ~[neo4j-kernel-2.2.9.jar:2.2.9]
at org.neo4j.kernel.extension.KernelExtensions.resolveDependency(KernelExtensions.java:112) ~[neo4j-kernel-2.2.9.jar:2.2.9]
This is from Neo4j 2.x (the Gremlin plug-in). The package, when built and deployed as instructed at https://github.com/thinkaurelius/neo4j-gremlin-plugin, does contain the jar-file which describes this class, and Maven did download it and did install it there. But, when the server attempts to load and execute the extension, nothing is resolved.
Why?

How to run appclient inside a deployed ear on wildfly?

I have an EAR-File containing an EJB-module, a WAR-module and two appclient-modules. I deployed the EAR using maven, so the EAR-File is not placed in the standalone/deployments-directory.
Connecting with jboss-cli.sh, I can see the deployment. The application is running and available via web browser.
Now I want to run one of the appclient-modules via command line. But I can not figure out how this is done, because I don't have the EAR-File available on the harddrive (it seems that an unzipped version is somewhere in standalone/tmp/vfs/...).
If the EAR was in standalone/deployments I would run:
./appclient.sh --host=127.0.0.1 path/to/myear.ear#appclient-module.jar
How can I run the appclient-module.jar contained by the deployed EAR? I tried using the deployment-name with the appclient.sh script:
./appclient.sh --host=127.0.0.1 deployment-name.ear#appclient-module.jar
But this results in:
java.lang.RuntimeException: JBAS013239: Could find application client /opt/wildfly-8.2.1.Final/bin/deployment-name.ear
at org.jboss.as.appclient.subsystem.Main.main(Main.java:135)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.jboss.modules.Module.run(Module.java:312)
at org.jboss.modules.Main.main(Main.java:460)
Thank you in advance!
It seems like there is no solution to the problem I described, so I came up with a workaround:
I am now using the maven-antrun-plugin to upload a copy of the deployed EAR-File. This second EAR-File is used in the cron-job.

Migrating Application from WAS 6.1 to WAS 8.5

I am migrating applications from WAS 6.1 to WAS 8.5. Existing application works fine in WAS 6.1 but in WAS 8.5 while connecting to a bpm wsdl url, I am getting the following exception. The jar is present in WEB-INF/lib
java.lang.NoClassDefFoundError: org.apache.woden.resolver.URIResolver
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:85)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:162)
at org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:65)
at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:815)
at org.apache.axis2.deployment.RepositoryListener.loadClassPathModules(RepositoryListener.java:222)
at org.apache.axis2.deployment.RepositoryListener.init2(RepositoryListener.java:71)
at org.apache.axis2.deployment.RepositoryListener.<init>(RepositoryListener.java:64)
at org.apache.axis2.deployment.DeploymentEngine.loadFromClassPath(DeploymentEngine.java:177)
at org.apache.axis2.deployment.FileSystemConfigurator.getAxisConfiguration(FileSystemConfigurator.java:135)
at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64)
at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContextFromFileSystem(ConfigurationContextFactory.java:210)
at org.apache.axis2.client.ServiceClient.configureServiceClient(ServiceClient.java:151)
at org.apache.axis2.client.ServiceClient.<init>(ServiceClient.java:144)
Switch the class loader to local first, parent last.
In application configuration go to:
Class loading and update detection
Change class loader order to local first
Save changes
Restart application server
For using third party Web service engine you need to set class loader to parent last and disable annotation scanning:
Set the DisableIBMJAXWSEngine property in the META-INF/MANIFEST.MF of
a WAR file or EJB module to true
See documentation for more details:
Using a third-party JAX-WS web services engine

Axis2 WS consumer in WebMethods8.2

I've got into a scenario where I have to get an Axis2 based ws consumer working within WebMethods as a java service. I've implemented the ws consumer first in netbeans just to see if it works and thus i found that the minimal amount of jars I'll require are the following:
[ xmlschema-1.4.7.jar, apache-mime4j-core-0.7.2.jar,
axiom-api-1.2.13.jar, axiom-impl-1.2.13.jar, axis2-adb-1.6.2.jar,
axis2-kernel-1.6.2.jar, axis2-transport-http-1.6.2.jar,
axis2-transport-local-1.6.2.jar, commons-codec-1.3.jar,
commons-httpclient-3.1.jar, commons-logging-1.1.1.jar,
httpcore-4.0.jar, mail-1.4.jar, neethi-3.0.2.jar, wsdl4j-1.6.2.jar ]
I've uploaded these jar files under the IS/packages/{package_name}/code/jars folder. Whenever I try to execute the java service that would send the request and process the response I get the following exception:
java.lang.reflect.InvocationTargetException:
org.apache.axiom.om.OMFactory.getMetaFactory()Lorg/apache/axiom/om/OMMetaFactory;
From the IS error log file I found that the actual error message is as follows:
org.apache.axiom.om.OMFactory.getMetaFactory()Lorg/apache/axiom/om/OMMetaFactory;
Caused by: java.lang.reflect.InvocationTargetException: null Caused
by:
java.lang.NoSuchMethodError:org.apache.axiom.om.OMFactory.getMetaFactory()Lorg/apache/axiom/om/OMMetaFactory;
The platform is WebMethods 8.2 under Linux environment. The JDK version is 1.6.0_32 and the application server under WebMethods is Jetty.
Actually the solve of this problem was a bit more tricky. First of all I manually had to configure the manifest file of the package on the IS server to use the jars provided in the package abnd thus it wouldn't get in conflict with the Axis used by the IS itself. On the other hand I had to manually add the ClassLoader because WebMethods can't use META-INF based information from jar files as it seems. To solve this problem simply use:
System.setProperty("org.apache.axiom.om.OMMetaFactory", "org.apache.axiom.om.impl.llom.factory.OMLinkedListMetaFactory");
That solves all the problems.

NoClassDefFoundError while deploying an application on WAS

I am trying to deploy an application on WAS 6.1 Server.
However, I am getting the below error:
java.lang.NoClassDefFoundError: org.apache.log4j.Logger
at myPackage.myClass.<clinit>(myClass.java:40)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:194)
at java.lang.Class.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1328)
at com.ibm.ejs.container.EJSContainer.loadBeanMetaData(EJSContainer.java:1756)
at com.ibm.ejs.container.HomeOfHomes.loadBeanMetaData(HomeOfHomes.java:1073)
at com.ibm.ejs.container.HomeRecord.getHomeAndInitialize(HomeRecord.java:454)
at com.ibm.ejs.container.EJSContainer.getHomeWrapperCommon(EJSContainer.java:1322)
at com.ibm.ejs.container.EJSContainer.getHomeInstance(EJSContainer.java:1231)
at com.ibm.ejs.container.EJSContainer.startBean(EJSContainer.java:1217)
at com.ibm.ws.runtime.component.EJBContainerImpl.startBean(EJBContainerImpl.java:3385)
at com.ibm.ws.runtime.component.EJBContainerImpl.startModule(EJBContainerImpl.java:2589)
at com.ibm.ws.runtime.component.EJBContainerImpl.start(EJBContainerImpl.java:3719)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1304)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1165)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:587)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:832)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:950)
at com.ibm.ws.runtime.component.ApplicationMgrImpl$AppInitializer.run(ApplicationMgrImpl.java:2131)
at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:342)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1473)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger
at com.ibm.ws.classloader.CompoundClassLoader.findClass(CompoundClassLoader.java:579)
at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java:414)
at java.lang.ClassLoader.loadClass(ClassLoader.java:597)
... 22 more
In
Admin Console -> Servers -> Application Servers -> myServerName -> Java and Process Management -> Process Definition -> Java Virtual Machine -> ClassPath
I have an entry as below:
${MY_JARS}\log4j-1.2.15.jar
Where in 'Custom Properties' MY_JARS key contains value as the path to the log4j jar
I did server clean and restart, republish, redeploy the application umpteen number of times. But this error is not going.
What am I doing wrong ?
To use ${MY_JARS} that way, you need to define it in Environment > WebSphere Variables, not in Custom Properties.
However I'd caution you about adding log4j in that manner. I seem to recall log4j might not work correctly if a single jar is shared across multiple apps. I think we attempted using log4j as a Shared Library (probably the more recommended way of doing what you're trying to do, BTW), but ended up needing to deploy to each application's WEB-INF/lib instead.
As dbreaux already pointed out you have to create environment variables with the correct scope. A custom property is available at runtime, but not for the server configuration level.
Within the WAS you can check what libraries are loaded. IIRC you have to click on the Application, afterwards on the Web Module and than you should be able to check the classloaders. If you open up the classloaders, you will see what libraries are loaded.
Please make sure log4j.jar is on the CLASSPATH of your application.For example; under WEB-INF\lib

Categories

Resources