Apache Samza does not run - java

I am trying to set up a Apache Samza and Kafka environment. I am experiencing some problems when trying to run the modules.
I have Kafka working correctly but I can not make Samza work. I have installed two Debian Jeesy AMD64 boxes and followed the instructions of the Samza documentation:
apt-get install openjdk-7-jdk openjdk-7-jre git maven
git clone http://git-wip-us.apache.org/repos/asf/samza.git
cd samza
./gradlew clean build
When I try to launch the script that should start the Yarn AppMaster with the script provided with Samza:
/opt/samza/samza-shell/src/main/bash/run-am.sh
I get this error:
Error: Main class org.apache.samza.job.yarn.SamzaAppMaster has not been found or loaded
If I try to run a test job with the run-job.sh script
./run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
and I get a similar error referencing the org.apache.samza.job.JobRunner class.
I am thinking that I have a java configuration issue, but I am not able to find much help or reference.
Does anyone know what I am doing wrong?

Still not working but I have gone one step ahead. When executing the Samza provided scripts from a path, they expect to be located in a /bin/ folder and they need to have a /lib/ one where all the samza .jar files should be located.
I am still having some dependencies issues, but different ones.

Related

Eclipse common launch for several machine

I have a project written in Java using Eclipse and I have a lot of users (students) that every day have to install and run it.
I'm searching a common configuration like npm start.
My idea is to install something or to say to my user: go in this directory and do this command for running like for Angular (npm install & npm start).
Is it possible or no?
Or to have a script that configure Eclipse workspace in automatically way.

How to Run Eclipse+Maven+JUnit tests in Command Line

When I run mvn clean test, it gives me the following error:
'mvn' is not recognized as an internal or external command, operable program or batch file."
This is the link of my project.
I can run the tests by right-clicking on the Runner.java => Run as => JUnit Test. I can also run the tests from the JUnit 4 Test Explorer Tab.
I have read some articles where it's suggesting to set the PATH for Maven but all of them are about the case where you are installing Maven separately. I am not sure what PATH do I need to set when Maven is coming with Eclipse.
I did not install Maven separately, it came with my Eclipse version.
My Eclipse Version :
Eclipse IDE for Java Developers
Version: 2019-03 (4.11.0)
Build id: 20190314-1200
Some IDEs (IntelliJ for example) have their own version of maven for internal purpose. But it doesn't mean that's able to be used by the command line. If you want to compile your project using mvn clean install/package/deploy/test/etc you must install maven in your machine and set de VARs in the OS.
I guess Eclipse has the same behavior that Intellij, so please also install maven in your machine
This error means that Windows cannot find an executable program called "mvn". This means it is not on your path/current directory.
When you right click and run your junit tests, it invokes it using the JVM, using the java command. I suspect if you type "java -v" into CMD, it will not give you the same error.
Since Windows does not know where to look for maven, you must tell it implicitly, by setting your path to the directory containing the mvn executable. You can lookup "eclipse bundled maven default location" to find this, although it might be worth downloading it separately from the Apache website, and manually specifying the path.

Cannot run Maven on Jenkins with Ubuntu

I am struggling to get a simple maven project running on Jenkins, which runs on my new Ubuntu server (16.04.)
I have installed Jenkins onto an Ubuntu, installed Maven and JDK.
I can connect to bitbucket and access my remote repository and get hold of my code. However, when I try to build I get the message
cannot run program "mvn" (in directory "/var/snap/jenkins/workspace/project") no such file or directory
I have added $JAVA_HOME and $MAVEN_HOME to my /etc/environment as variables and also into my path.
I have looked at the Global Tool Configuration, but this seems to be no help.
When I set the Maven installer to point at Maven home I get a message
/usr/share/maven is not a directory on the jenkins master
When I try to get Jenkins to install maven for me I get
Failed to install <truncated>apache-maven-3.5.3.bin.zip to /var/snap/jenkins/6/tools
...
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must not be empty
I think I have hit a dead end. What could I be missing, or how can I get past these obstacles ?
Many Thanks
Go to Manage Jenkins -> Global Tool Configuration. I always elect to have Jenkins install and manage Maven, vs install and bring my own.
It has taken a long time to get to the bottom of this.
By default the
sudo apt-get install jenkins
seems to install a Snap version of Jenkins.
This snap version holds data in different locations (which is why I struggled to find the initialAdminpassword
Additionally, the snap setup only has a JRE, which cannot be overridden to a JDK, and therefore does not seem to work properly with Maven.
Eventually, I completely uninstalled Jenkins, and then followed the instructions on the official Jenkins site:
https://pkg.jenkins.io/debian-stable/
This overrides the base apt-get packages, and points to the jenkins.io packages.
From here on in, it all behaved as per the documentation.

Execute JUnit tests inside Docker container

I would like to build a test environment with Docker, where I can remotely send JUnit test classes (including the code that is tested), execute the tests and retrieve the results.
I found some articles which explained how to use docker for testing databaseconntection/writing inside a redis, but not how i can simple let my tests perform on docker and retrieve the results.
Do you have any recommendations how You would actually achieve this?
I don't know much about Jenkins, but would this might solve my problem?
Is there any good framework outside for this?
In a dockerfile, checkout your code and do a "maven test" command, redirect the result in a file that is on a mounted directory.
Each time you build the dockerfile, you do a unit test on your project.
With docker you also have a "docker test" command. I dont know if there is a plugin to use it on jenkins.
One way I found that works (using Gradle) is as follows. I know you are specifically referencing JUnit as your testing framework, but I actually think something similar to this could work.
Dockerfile (I called mine Dockerfile.UnitTests):
FROM gradle:jdk8 AS test-stage
WORKDIR /app
COPY . ./
RUN gradle clean
RUN gradle test
FROM scratch AS export-stage
COPY --from=test-stage /app/build/reports/tests/test/* /
I then run this with (in Gitbash on Windows 10):
> DOCKER_BUILDKIT=1 docker build -f Dockerfile.UnitTests --output type=tar,dest=UnitTests.tar .
This results in a tar file containing the test results displayed in an html file.
I executed the above in a Gitlab CI/CD pipeline and then sent the results to a web API for analysis.
A couple of assumptions:
My project is set up for Gradle builds so I have the structure from the root of my project src/test/java/groupname/projectname/testfile.java
I am working in Windows 10 targeting Linux containers and using Gitbash.

Java run time error when running the jar created from Maven (3.0.5) build using ess_es_server, ess_japi jar

I have a strange issue; I'm trying to run the jar created from maven build thru command prompt, but it gives run time error. When I'm running this from eclipse, it runs fine. Also, build is successful. Here's what I'm doing:
Created a java swing project (java 1.7) and tested successfully in eclipse
pushed the code to git for others to review and to be merged to master
once I got the go ahead after some minor changes (comments only), cloned it to a dev folder and keep it separate
ran mvn archetype:generate and create a mvn project w/ pom.xml
import this project to eclipse and retrofit the clean code from dev folder
modified pom.xml for dependency on the 3rd party jars (ess_es_server, ess_japi) and saved in eclipse.
came back to cmd prompt;compiled, packaged to create the jar- it was successful
while launching the app (java -cp < cp with jar) pkg.app, it gives error
“…java.lang.NoClassDefFoundError: com/essbase/api/base/EssException” whenever it's trying to invoke that particular class pertaining to one of the 3rd party jars
I have installed these 3rd party jars to my .m2 repo since they are not available in public repository.
Can someone help? My suspicion is my GAV parameters are not correct.
While running explicitly mention the library from where java program should pick up the jars; .m2 repo is used for compilation time only.

Categories

Resources