Jenkins file system on Linux server VS Jenkins EXECUTOR_NUMBER variable - java

I have Jenkins on Centos server with only one job calls "HOMEPAGE".
I would like to run this job in parallel, this is why I setup 5 executors for master node.
This "HOMEPAGE" job running java program that creates some folders and files that I need to use. So I need to know the fool path to files on server that was creating during this job.
Problem start when I run this job in parallel.
For first build Jenkins would assigne path: /var/lib/jenkins/workspace/HOMEPAGE/ and executor_number=2
For second build: /var/lib/jenkins/workspace/HOMEPAGE#2/ and executor_number=4
For third: /var/lib/jenkins/workspace/HOMEPAGE#3/ and executor_number=1
For fourth: /var/lib/jenkins/workspace/HOMEPAGE#4/ and executor_number=3
After execution I could see this folders on server:
As you can see, the number of the HOMEPAGE folder is not consistent with executor_number variable in Jenkins.
How could I get information from Jenkins about where it saving results in current build? Whether it HOMEPAGE#2 or HOMEPAGE#4 folder? I need this information for my java program.
Here is fragments from Console Output:
First build:
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven: -B -f /var/lib/jenkins/workspace/HOMEPAGE/pom.xml -PHomepage -Djob_name=HOMEPAGE -Dexecutor_number=2
Third build:
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven: -B -f /var/lib/jenkins/workspace/HOMEPAGE#3/pom.xml -PHomepage -Djob_name=HOMEPAGE -Dexecutor_number=1

The environment variable 'WORKSPACE' will always contain the proper path, including any #<n> suffix.

Related

How to automatically start 2 Java jars on AWS EC2?

I'm learning to deploy Spring Boot apps on AWS EC2. And I know how to automate app launch, when I start the EC2 instance, I don't need to manually use the command java -jar java-service.jar, I just add this command in the /etc/rc.local file and that is all. But I have 2 microservice, and I want to start both of them automatically, but if I try to add both commands in the /etc/rc.local it's not working, only the first service will start, the second service will not start.
So I have the commands added like this:
And after I start the EC2 instance only the first service is started:
Thank you!
I am not a unix expert, but I see the only issue in running 2 java commands from terminal is that unless the first command returns, the next command is not executed. So, I think the solution would be run the 1st command in some interactive mode so that the other commands can be executed simultaneously.
There are ways in unix shell to run a command in background. I found this useful link - https://www.maketecheasier.com/run-bash-commands-background-linux/
In bash terminal, a command can be made to run in background by appending it with &. So, I think you should be able to start both jars if you do something like -
java -jar /home/ec2-user/first.jar &
java -jar /home/ec2-user/second.jar
I recommend to use Systemd.
Create a Systemd unit file for every microservice, save it in /etc/systemd/system/my-app.service. Something like that:
[Unit]
Description=My Java app
After=syslog.target network.target
[Service]
EnvironmentFile=/etc/sysconfig/my-app-env
WorkingDirectory=/my/app/home
ExecStart=/usr/bin/java $JAVA_OPTS -jar my-app.jar
KillMode=process
User=my-app-user
Restart=on-failure
[Install]
WantedBy=multi-user.target
Then, run:
systemctl daemon-reload
systemctl enable --now my-app
After that, you can use:
systemctl status my-app
systemctl stop my-app
systemctl start my-app
Another solution is to bundle your jars into Docker images. This of course requires Docker runtime and adds an overhead, but it also has some benefits:
Complete separation of jar files. Easily use different java versions.
No need to worry about differences of local and ec2 environment.
Easily scale to 3 or more jars.
Use Docker Cli to build and start containers. Works great in a Devops Pipeline.
You can read here to learn how to create Spring Boot Docker images. After you build an image. You start it like this.:
docker run -p 8080:8080 springio/gs-spring-boot-docker
You can run as many docker run commands you need, one after another.
I am not sure which system you are using in starting application:
For linux base system, you can use crontab to schedule the task when the server reboot.
Follow this steps:
Download crontab
#apt-get install cron
Edit the file file to enable the task
crontab -e
(Choose Vim or nano to edit the task)
Add this code to your server
#reboot /usr/bin/java -jar XXXXX.jar
Save your file
Check the result
crontab -l
#systemctl status cron
This method works in my Debian system. For more details, you can refer to
How to automatically run program on Linux startup
If you are running from bash, then join two jar commands with "&" like below.
java -jar /home/ec2-user/first.jar&java -jar /home/ec2-user/second.jar
coupon service
Run the command 'java -jar /home/ec2-user/coupon-service-0.0.1-SNAPSHOT.JAR'
Press CTRL+Z, type bg, press Enter, type disown, press Enter.
product service
Run the command 'java -jar /home/ec2-user/product-service-0.0.1-SNAPSHOT.JAR'
Press CTRL+Z, type bg, press Enter, type disown, press Enter.
NOTE: Both services should have different ports.

Apache airflow task: java: command not found

I'm running Apache Airflow using Docker. I have installed OpenJDK12 and defined its variables in both of:
~/.bachrc
/usr/local/airflow/.bachrc
export JAVA_HOME=/usr/local/src/jdk-12
export PATH=$PATH:$JAVA_HOME/bin
and I can see Java is running normally especially when I run the Task from the command line :
airflow test ip-importer Download 2020-2-19
where it's working properly without errors , where the task is for :
Executing java command to run a Jar file
But now when I schedule the task on the webserver UI and when I run it, It doesn't run properly and gives me this error:
[2020-02-18 00:46:39,941] {{bash_operator.py:126}} INFO -
/tmp/airflowtmpdzijcupu/Download886rmbol: line 1: java: command not
found
so seems like It's unable to see the defined paths for Java.
Any hints / ideas for how to solve this issue ?
Another question , In case if the java application has thrown an exception/error , Airflow doesn't assume it as an error , it assumes it as a succeeded task , Could you advise how to enforce airflow to assume the exception as a real interrupter for the task ?
Check if /usr/bin folder from Docker container has the executable file pointing out to Java cmd. If not, create one with the following cmd:
Example:
ln -s /usr/local/airflow/java/jdk1.8.0_161-x64/bin/java /usr/bin/java

Restore old Jenkins jobs in Windows after the update

We were running plug-in rich Jenkins 2.46.1 and one of my team member tried to update Jenkins but it just hangs on displaying Please wait while Jenkins is restarting message for about 2 hours so I somehow forced Jenkins to shutdown and then started it using java -jar Jenkins.war command.
When Jenkins restarted again all my jobs were not displayed in Jenkins GUI but they were present in jobs and workspace folders so I tried an option Reload configuration from disk but that also did not restore jobs.
Please someone advice how I can restore my Jenkins jobs as they were before.
I believe you are not pointing JENKINS_HOME correctly. You may need something like this as startup script/batch file
For Linux:
export JENKINS_HOME=<path/to/old/jenkins/home>
java -jar Jenkins.war
For Windows:
set JENKINS_HOME=<path/to/old/jenkins/home>
java -jar Jenkins.war
Basically running war directly points to default temp directly for Jenkins Home. It can be overwritten by above environment variable. You can also centrally define it so that each start you do not need specify again.

Unable to run java jar command on target machine in Ansible

I am trying to run the following command:
java -jar jar-from-application.jar --file myown.properties
Couple of problems I faced earlier while trying to run this on target machine. Tomcat needs to be running and a war file needs to be deployed completely before I run the above command. This is what I did to insure that pre conditions are met. For Tomcat:
nohup startup.sh
and then check for deployed file bu using curl to check for status 200:
- name: Check if URL is available
shell: curl --head --silent http://target-mchine:8080/restoftheurl
register: result
until: result.stdout.find("200 OK") != -1
retries: 12
delay: 10
It does above two steps correctly but when the next task is to run the
java -jar jar-from-application.jar --file myown.properties
it for some reason says that it cannot read the properties file. However, if I run the same jar command on the remote machine locally, then it works perfectly. I even added some wait time after successful execution of curl command, just in case, but it doesn't make any difference. I tried doing the following too:
nohup java -jar jar-from-application.jar --file myown.properties
as it was suggested online but that didn't make any difference either. Any suggestions will be appreciated. Thank You.
Make sure that you set the correct path to the properties file in the command by using the chdir argument of the command module or set the full path in the command.

Running Jmeter testplan in non GUI mode

I have created some test plans in Jmeter. Now I need to run them trough command line or java API.
Can anyone please suggest any links or examples on from CLI/JAVA API how to:
Start jmeter
Load the *.jmx [testplan] plan
Specify number of threads
Start the test
Redirect the output result xml to result directory.
In addition to previous comment on how to run JMeter in non-GUI mode, number of threads can be passed as JMeter property as follows:
In Thread Group set "Number of Threads" to be ${__property(users,,)}
and set it as
jmeter -Jusers=50 -n -t Test_Plan.jmx -l results_folder\log.jtl
See Apache JMeter Properties Customization Guide for more details.
In regards to running JMeter test from Java code refer to this thread.
For running JMeter scripts from Apache Ant there is JMeter Ant Task
For running JMeter by Maven there is a JMeter Maven plugin
There is also Jenkins plugin if you want to integrate it with Jenkins/Hudson
Now i need to run them trough command line or java API
For running a jmeter test plan through CLI, you want a couple flags --
jmeter -n -t Test_Plan.jmx -l log.jtl
-n - Non gui mode
-t location of the test plan
-l log file to output
Specify number of threads
This is actually built into the test plan - you would want to edit the .jmx test plan to change the number of threads.

Categories

Resources