Develop Reference

Datadog instrumentation on Spring boot container application - class not found

I'm currently running a spring boot application as container into a kubernetes cluster.
Datadog agent is running as containers on the cluster.
I have modified the container image build to include the datadog agent before running the application:
CMD ["/bin/sh","-c","java -javaagent:dd-java-agent.jar -Ddd.trace.config=datadog.properties -Dlogging.config=/logback.xml -jar service.jar"]
I also setup the environment variable to indicate the HOST IP of the agent to my container via the Deployment file.
The problem now is i'm getting this class not found exception when the application starts:
java.lang.NoClassDefFoundError: datadog/trace/instrumentation/springscheduling/SpringSchedulingInstrumentation
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at org.springframework.core.annotation.AnnotationsScanner.processClassHierarchy(AnnotationsScanner.java:233)
at org.springframework.core.annotation.AnnotationsScanner.processClassHierarchy(AnnotationsScanner.java:194)
at org.springframework.core.annotation.AnnotationsScanner.processClass(AnnotationsScanner.java:128)
at org.springframework.core.annotation.AnnotationsScanner.process(AnnotationsScanner.java:107)
at org.springframework.core.annotation.AnnotationsScanner.scan(AnnotationsScanner.java:97)
at org.springframework.core.annotation.AnnotationsScanner.scan(AnnotationsScanner.java:78)
at org.springframework.core.annotation.TypeMappedAnnotations.scan(TypeMappedAnnotations.java:242)
at org.springframework.core.annotation.TypeMappedAnnotations.get(TypeMappedAnnotations.java:149)
at org.springframework.core.annotation.TypeMappedAnnotations.get(TypeMappedAnnotations.java:131)
at org.springframework.core.annotation.AnnotationUtils.findAnnotation(AnnotationUtils.java:581)
at org.springframework.cloud.sleuth.DefaultSpanNamer.annotation(DefaultSpanNamer.java:71)
at org.springframework.cloud.sleuth.DefaultSpanNamer.name(DefaultSpanNamer.java:58)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.<init>(TraceRunnable.java:59)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.<init>(TraceRunnable.java:51)
at org.springframework.cloud.sleuth.instrument.async.LazyTraceThreadPoolTaskScheduler.scheduleWithFixedDelay(LazyTraceThreadPoolTaskScheduler.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.cloud.sleuth.instrument.async.ExecutorMethodInterceptor.invoke(ExecutorBeanPostProcessor.java:328)
Quite straigtforward, i need to include some dependencies into the application package. But i could not find anything useful on datadog website nor maven central repository.
Including the agent itself or the api libraries fix nothing. This class is present on the agent but under a different path.
Does anybody know which dependencies should be included in the classpath of the application to fix that ?

Spring is scanning your classpath that seems incomplete. Maybe it is related to Spring's class loading mechanisms. The class in question exists and seems to be part of the agent.
Possibly, you are using an outdated version of the agent.

Multiple applications sharing common EJB JAR in TomEE

I have an EJB jar DetailsLookup.jar which needs to be shared by two consumer applications - VehicleMake.war and VehicleModel.war
I have placed the EJB jar file under <catalina_base>/lib and both the war file under <catalina_base>/webapps
When I deploy only one of the WAR files, the deployment is successful, but with 2 WAR files I get the error as below. It seems, TomEE is trying to deploy the EJB for every consumer application instead of deploying it only once
Caused by: org.apache.openejb.DuplicateDeploymentIdException: Application cannot be deployed as it contains deployment-ids which are in use: app: D:\apache-tomcat\instance1\webapps\VehicleModel
DetailsLookup
at org.apache.openejb.assembler.classic.Assembler.createApplication(Assembler.java:790)
at org.apache.openejb.assembler.classic.Assembler.createApplication(Assembler.java:756)
at org.apache.tomee.catalina.TomcatWebAppBuilder.startInternal(TomcatWebAppBuilder.java:1308)
... 32 more
I have also noticed that when I deploy only VehicleMake.war, the JNDI name for my EJB has the application context path also appended (which says that the EJB is deployed per project and not common)
Expected JNDI name:
global/DetailsLookup!com.my.company.details.service.DetailsLookup
Actual JNDI name:
global/VehicleMake/DetailsLookup!com.my.company.details.service.DetailsLookup
Can anyone help me to identify what the issue is with deploying EJB and if I need to change any configurations to deploy it as a common EJB instead of application specific?

In conf/system.properties, you can change the name of deployed artifacts to avoid collisions.
Try adding this line:
openejb.deploymentId.format = {appId}/{ejbJarId}/{ejbName}

WebSphere7 : java.lang.NoSuchMethodError: javax/persistence/spi/PersistenceUnitInfo.getSharedCacheMode()Ljavax/persistence/SharedCacheMode;

Hello everyone long story short,i'm using hibernate in my project,i try to deploy my project on WebSphere but it's giving me the error you see in the question title
,as you see in my jars there is no trace of a lower version of jpa 2.0
jars included in my project
but i know that all servers have a version of jpa,and websphere 7 is using 1.0 that i found among it's jars,here is a picture of it,
WebSphere 7 jars
if anyone please knows how to resolve this problem or found this problem before i will be so thankful,thank you in advance, i will leave the full stack trace for more infos :
Caused by: java.lang.NoSuchMethodError: javax/persistence/spi/PersistenceUnitInfo.getSharedCacheMode()Ljavax/persistence/SharedCacheMode;
at org.hibernate.ejb.util.LogHelper.logPersistenceUnitInfo(LogHelper.java:39)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:516)
at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:73)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:288)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:310)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1547)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1485)
... 61 more

I had a similar problem deploying my application with jpa 2.1 in Websphere 7.
If is the same problem, you have to tell Websphere to use Hibernate persistance provider instead of Default Persistance provide.
I wrote a guide with the solution that worked for me in this answer:
https://stackoverflow.com/a/38505479/6503002
EDIT: (Explain step 1 and 2 of the guide)
Step 1: To create a folder you have to access to server machine (through ssh) and create it in the file system (e.g. /usr/sharedHibernate assuming your server runs on linux). Then copy hibernate libraries in the folder you created (from your local machine to the server, with Filezilla or WinScp or Putty).
Step 2: In Websphere console on the left you have Environment section that contains Shared Libraries subsection:
Here create a new shared library, give it the name you want and in classpath textarea insert the path of the folder you created before (e.g. /usr/sharedHibernate)

Hibernate java.lang.ClassCastException: _$$_javassist_856 cannot be cast to javassist.util.proxy.Proxy when using Websphere Shared Library

Websphere 8.0.0.11
Hibernate 4.2.21.Final
I have found many questions about this same problem but none of them worked for me.
If I deploy the application in Websphere it works OK.
However we have defined a shared library that contains all the third party libraries (spring, hibernate, javassist, etc) so that our WARs are thinner.
This way during deployment we associate our thin WAR against that Websphere shared library.
The point is that when we deploy the application this way the ClassCastException Hibernate exception _$$_javassist_856 cannot be cast to javassist.util.proxy.Proxy is thrown.
I have checked the loaded jars in the websphere console and can only see one javassist jar (3.18.1-GA) in the classpath.
Why could this be happening?
UPDATE
I have also tried using PARENT_FIRST and PARENT_LAST class loading.
UPDATE 2
I just found out that Websphere is loading its own javassist jar:
URL location = ProxyFactory.class.getProtectionDomain().getCodeSource().getLocation();
logger.info("{}", location);
It prints: file:/opt/IBM/WebSphere/AppServer/plugins/javassist.jar

After trying pretty much everything I found on S.O. without any success I decided to downgrade Hibernate to version 4.1.12.Final. This is the maximum 4.x version compatible with Websphere 8.x.

The problem is that Javassist leaves traces in its generated code. With Javassist on the class path twice, its classes are loaded twice. Two types are however only equal if they have the same name and are loaded by the same class loader. In your case, the generated class resolves its Javassist dependeny to a type that is loaded by your application class loader while your code is casting the instance to the Javassist type that is loaded by the Websphere class loader (or the other way around).
Are you sharing any Hibernate dependencies between applications? Try to not use any shared libraries related to Hibernate in your application to avoid this.

SOAP error : Axis 2 AbstractMethodError

I have problem with my webservice client which build using Axis 1.3.1 I
encounter below error when executing the webservice:
[9/5/14 15:05:47:211 EDT] 0000018f ServletWrappe E com.ibm.ws.webcontainer.servlet.ServletWrapper service Uncaught service() exception thrown by servlet SWPChooseAccountServlet: java.lang.AbstractMethodError: org/apache/axis2/databinding/ADBBean.serialize(Ljavax/xml/namespace/QName;Ljavax/xml/stream/XMLStreamWriter;)V
at org.apache.axis2.databinding.ADBDataSource.serialize(ADBDataSource.java:90)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.internalSerialize(OMSourcedElementImpl.java:781)
at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:967)
at org.apache.axiom.soap.impl.llom.SOAPEnvelopeImpl.serializeInternally(SOAPEnvelopeImpl.java:283)
at org.apache.axiom.soap.impl.llom.SOAPEnvelopeImpl.internalSerialize(SOAPEnvelopeImpl.java:245)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.serializeAndConsume(OMSerializableImpl.java:207)
at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:74)
at org.apache.axis2.transport.http.AxisRequestEntity.writeRequest(AxisRequestEntity.java:84)
The webservice client is deployed in IBM WAS 8.5.5.1 My temporary solution is
to add axis2-adb-1.4.1.jar to WebSphere app server shared library and add
the library as a reference library in the Java class loader (Classes loaded
with local class loader first (parent last))

Thanks
It works.
Adding Shared Libraries For axix2 in WAS 8.5
Login into WAS admin console.
Go to the Environment Link
Click on Shared libraries
Click on new; Select the scope of this variable.
Provide the name and class path value as per the library location of jars.
Press apply and then save button.
Now click on Servers > webSphere Appliacatio servers.
With in the list of servers select your server.
Click on Java and Process Management > Class Loader
Go new and select “Class loader order” and choose “Classes loaded with local class loader first (parent last)”.
Apply and save.
Now Click on Shared library references under “Additional Properties” section and include your added shared library variable name.
Now restart server instance to see the effect of whole task.