Develop Reference

How to use Apache Tattletale to analyze duplicate Jar/APIs used in class path

In my project they have used more than 225+ jar files which causing memory issue, while searching on net i come to know Apache Tattletale will analyze and give a report of duplicate classes and JAR/APIs used by the application (Classpath). So i have refereed following links
1) how to use JBoss Tattletale tool
2) Uncover JBoss client jar list with Tattletale
3) Jboss official Documentation
but i didn't get how to execute and run the Tattletale Jar file and my application is not based on maven so i am not using Maven.
I have downloaded tattletale-1.2.0.Beta2.jar file along with jboss-seam-2.3.0.CR1-dist file and used following command
java -Xmx512m -jar tattletale.jar /Java/workspaces/mycoolprojects/projectX output-projectx
but getting following exception
Exception in thread "main" java.lang.NoClassDefFoundError: javassist/NotFoundException
at org.jboss.tattletale.analyzers.Analyzer.getScanner(Analyzer.java:49)
at org.jboss.tattletale.Main.execute(Main.java:608)
at org.jboss.tattletale.Main.main(Main.java:1099)
Caused by: java.lang.ClassNotFoundException: javassist.NotFoundException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
More over i didn't get what is the use of jboss-seam-2.3.0.CR1-dist file. Meaning i can see lot of jar files and lot of code in there but i don't know how does it help to use tattletale.
In the official documentation they have mentioned jboss-tattletale.properties and how can i set/use that.

I was having the same problem and this solution worked for me too.
(downloaded the latest javaassist jar)
Interestingly, tattletale itself suggests that the tattletale jar contains the
javaassist jar

The below steps worked for me:
download jboss-javassist-javassist-rel_3_22_0_cr1-2-g6a9079a.zip from http://jboss-javassist.github.io/javassist/
extract it to a location
go to that location and copy javassist.jar
go to location where your tattletale-1.2.0.Beta2.jar is present
paste javassist.jar here
open command prompt at this path
run command java -jar tattletale-1.2.0.Beta2.jar path_to_application_archive output_path

I inherited an old Maven project configured to use this plugin and got the same javassist errors. The plugin dependencies may be adjusted as shown to make the errors stop.
<plugin>
<groupId>org.jboss.tattletale</groupId>
<artifactId>tattletale-maven</artifactId>
<version>1.2.0.Beta2</version>
<executions>
<execution>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- This is the location which will be scanned for generating tattletale reports -->
<source>${project.build.directory}/${project.artifactId}/WEB-INF/lib</source>
<!-- This is where the reports will be generated -->
<destination>${project.build.directory}/site/tattletale</destination>
</configuration>
<dependencies>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<version>3.27.0-GA</version>
</dependency>
</dependencies>
</plugin>

Running app jar file on spark-submit in a google dataproc cluster instance

I'm running a .jar file that contains all dependencies that I need packaged in it. One of this dependencies is com.google.common.util.concurrent.RateLimiter and already checked it's class file is in this .jar file.
Unfortunately when I hit the command spark-submit on the master node of my google's dataproc-cluster instance I'm getting this error:
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.createStarted()Lcom/google/common/base/Stopwatch;
at com.google.common.util.concurrent.RateLimiter$SleepingStopwatch$1.<init>(RateLimiter.java:417)
at com.google.common.util.concurrent.RateLimiter$SleepingStopwatch.createFromSystemTimer(RateLimiter.java:416)
at com.google.common.util.concurrent.RateLimiter.create(RateLimiter.java:130)
at LabeledAddressDatasetBuilder.publishLabeledAddressesFromBlockstem(LabeledAddressDatasetBuilder.java:60)
at LabeledAddressDatasetBuilder.main(LabeledAddressDatasetBuilder.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
It seems something happened in the sense of overwriting my dependencies. Already decompiled the Stopwatch.class file from this .jar and checked that method is there. That just happened when I ran on that google dataproc instance.
I did grep on the process executing the spark-submit and I got the flag -cp like this:
/usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp /usr/lib/spark/conf/:/usr/lib/spark/lib/spark-assembly-1.5.0-hadoop2.7.1.jar:/usr/lib/spark/lib/datanucleus-api-jdo-3.2.6.jar:/usr/lib/spark/lib/datanucleus-rdbms-3.2.9.jar:/usr/lib/spark/lib/datanucleus-core-3.2.10.jar:/etc/hadoop/conf/:/etc/hadoop/conf/:/usr/lib/hadoop/lib/native/:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*
Is there anything I can do to solve this problem?
Thank you.

As you've found, Dataproc includes Hadoop dependencies on the classpath when invoking Spark. This is done primarily so that using Hadoop input formats, file systems, etc is fairly straight-forward. The downside is that you will end up with Hadoop's guava version which is 11.02 (See HADOOP-10101).
How to work around this depends on your build system. If using Maven, the maven-shade plugin can be used to relocate your version of guava under a new package name. An example of this can be seen in the GCS Hadoop Connector's packaging, but the crux of it is the following plugin declaration in your pom.xml build section:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>your.repackaged.deps.com.google.common</shadedPattern>
</relocation>
</relocations>
</execution>
</execution>
</plugin>
Similar relocations can be accomplished with the sbt-assembly plugin for sbt, jarjar for ant, and either jarjar or shadow for gradle.

File not found exception, only when including the code as a maven dependency

I have written this project which has a bunch of config files in config/ folder, which are necessary for running:
https://github.com/danyaljj/jwnl-prime/
The code is working fine under mvn test in my computer and CI.
But after packaging it and adding it as a maven dependency to another project, the function calls give me error
java.io.FileNotFoundException: config/file_properties.xml (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at net.didion.jwnl.TestDefaults.getInputStream(TestDefaults.java:63)
at net.didion.jwnl.JWNL.initialize(JWNL.java:92)
Any idea why I am getting this weird behavior?
Update:
after extracting the jar file, it turns out that the config files are not included in the jar:
Update2: After adding the suggestion by ? the config files get included in the jar file:
But still getting the same error: config/file_properties.xml (No such file or directory)

If you have project structure other than standard format you need to update pom.xml as well. If you need to include the config for build you need to add that in build section like
Refer link enter link description here
<build>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh</artifactId>
<version>2.4</version>
</extension>
</extensions>
<resources>
<resource>
<directory>config</directory>
<includes>
<include>**/**</include>
</includes>
</resource>
</resources>
</build>

First, verify that the configuration files are actually being packaged into the jar. I would view it in an archive viewer or unpack it and see what's in it.
Secondly, you can't directly load a file from a jar using standard file opening methods. You'll have to use something like
Class.getResourceAsStream()
See here for some more suggestions:
How do I access a config file inside the jar?

Maven rpm plugin tries to run installation script on build

I am trying to create an RPM package to install a piece of software however whenever I try to build it using the rpm plugin it will run the install script while building which will fail since my machine is not the intended target (nor should it be)
The setup is this
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.1.2</version>
<extensions>true</extensions>
<configuration>
<group>Applications/Software</group>
<mappings>
<mapping>
<directory>/tmp/${project.artifactId}</directory>
<filemode>755</filemode>
<username>user</username>
<groupname>group</groupname>
<sources>
<source>
<location>src/main/resources/</location>
</source>
</sources>
</mapping>
</mappings>
<requires>
<require>unzip</require>
</requires>
<preinstallScriptlet>
<scriptFile>src/main/scripts/preinstall.sh</scriptFile>
<fileEncoding>utf-8</fileEncoding>
</preinstallScriptlet>
<installScriptlet>
<scriptFile>src/main/scripts/install.sh</scriptFile>
<fileEncoding>utf-8</fileEncoding>
</installScriptlet>
</configuration>
</plugin>
I have also configured the pom packaging to be rpm and I am running "mvn clean package" to generate the rpm.
This is just a builder project meaning that all it is meant to do is to package all files within src/main/resources in the rpm together with scriptlets which will execute when that rpm is run on some target machine.
Am I missing something?
I am building the rpm on an Ubuntu 14.04 machine with rpmbuild installed

I just read the RPM documentation and found out that this is totally correct. The install scriptlet is called when the RPM is build, what you probably need is a preinstall or postinstall scriptlet. The installation itself (copying the files) is done by RPM.
Reference: http://www.rpm.org/max-rpm/s1-rpm-inside-scripts.html

Heroku New Relic Add-on for Java App fails to open newrelic.jar

I downloaded New Relic 2.12.0 and configured it according to the Heroku java configuration documentation.
I unzipped the package contents to the root of my spring mvc application directory as indicated in the following image
Next, I pushed the new files to Heroku:
$ git add newrelic
$ git commit -m 'add newrelic'
$ git push heroku master
Finally, I bootstrapped the new relic agent:
$ heroku config:add JAVA_OPTS='-Xmx384m -Xss512k -XX:+UseCompressedOops -javaagent:newrelic/newrelic.jar'
Afterwards, my application failed with the following error and appears to not be able to locate the newrelic.jar file.
The new relic documentation says the JVM args (e.g. -javaagent:newrelic/newrelic.jar) should include the full path to the newrelic.jar file, but according the the Heroku documentation, -javaagent:newrelic/newrelic.jar is all that's needed.
2013-01-15T19:41:11+00:00 heroku[web.1]: Starting process with command `java -Xmx384m -Xss512k -XX:+UseCompressedOops -javaagent:newrelic/newrelic.jar -Dspring.profiles.active=prod -jar target/dependency/webapp-runner.jar --port 47412 target/*.war`
2013-01-15T19:41:11+00:00 app[web.1]: agent library failed to init: instrument
2013-01-15T19:41:11+00:00 app[web.1]: Error occurred during initialization of VM
2013-01-15T19:41:11+00:00 app[web.1]: Error opening zip file or JAR manifest missing : newrelic/newrelic.jar
2013-01-15T19:41:12+00:00 heroku[web.1]: Process exited with status 1
2013-01-15T19:41:12+00:00 heroku[web.1]: State changed from starting to crashed
UPDATE
After checking the newrelic directory on my web dyno, I noticed the jar file was missing. Now I am looking into the cause of the missing jar file.
Update
The .gitignore file in the root of my application directory excludes jar files which prevents the newrelic.jar file from being deployed.
<<<<<<< HEAD
/target
/.classpath
/.project
/.settings
/tomcat.*
/.idea
/*.iml
=======
*.class
# Package Files #
*.jar
*.war
*.ear

Is it possible you've left something in your .gitignore file that ignores jars? If you heroku run bash and then ls newrelic do you see the jars in there?

Had the same problem with the missing jar file in the target/dependency folder .
For me the problem was the pom file (provided by the Heroku example)
Here is my working POM:
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-runner</artifactId>
<version>9.2.3.v20140905</version>
<destFileName>jetty-runner.jar</destFileName>
</artifactItem>
<artifactItem>
<groupId>com.newrelic.agent.java</groupId>
<artifactId>newrelic-agent</artifactId>
<version>3.12.0</version>
<destFileName>newrelic-agent.jar</destFileName>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
And for sure also add New Relic as dependency...
<dependency>
<groupId>com.newrelic.agent.java</groupId>
<artifactId>newrelic-api</artifactId>
<version>3.12.0</version>
</dependency>