Develop Reference

Github action, maven build fails when using the

I try to CI/CD for the jave open source project, which is hosted on github.
When I build the project with maven local, then it works just fine.
But the same maven build fails when triggered inside the github/gitaction environment when building the jave-core target.
https://github.com/a-schild/jave2/blob/develop/jave-core/pom.xml
The special thing, is that I use the org.codehaus.mojo buildnumber-maven-plugin and also the org.codehaus.mojo templating-maven-plugin.
I think it fails because of this, but I am unable to find how to fix it.
Here is the gitaction
https://github.com/a-schild/jave2/blob/master/.github/workflows/maven.yml
And here the error log of the build
[INFO] Executing: /bin/sh -c cd '/home/runner/work/jave2/jave2/jave-core' && 'git' 'log' '-n1' '--date-order'
[INFO] Working directory: /home/runner/work/jave2/jave2/jave-core
[INFO] Executing: /bin/sh -c cd '/home/runner/work/jave2/jave2/jave-core' && 'git' 'pull' 'https://github.com/a-schild/jave2.git'
[INFO] Working directory: /home/runner/work/jave2/jave2/jave-core
Error: Provider message:
Error: The git-pull command failed.
Error: Command output:
Error: From https://github.com/a-schild/jave2
* branch HEAD -> FETCH_HEAD
hint: You have divergent branches and need to specify how to reconcile them.
hint: You can do so by running one of the following commands sometime before
hint: your next pull:
hint:
hint: git config pull.rebase false # merge
hint: git config pull.rebase true # rebase
hint: git config pull.ff only # fast-forward only
hint:
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
fatal: Need to specify how to reconcile divergent branches.

Travis CI + Coverity scan with Gradle

I've successfully setup a project which uses Travis CI to for builds and tests. Now I'm trying to add Coverity Scan.
I created a branch called coverity_scan and set it be used for coverity builds. After I push a commit to this branch I can see in Travis CI build console that Coverity tool starts doing its job:
Coverity Scan analysis selected for branch coverity_scan.
Coverity Scan analysis authorized per quota.
...
Running Coverity Scan Analysis Tool...
The Travis build succeeds and in Coverity build-log.txt file I see this:
2016-10-06T21:02:39.132946Z|cov-build|2665|info|>
2016-10-06T21:02:39.132946Z|cov-build|2665|info|> Build time (cov-build overall): 00:01:36.812431
2016-10-06T21:02:39.132946Z|cov-build|2665|info|>
2016-10-06T21:02:39.134719Z|cov-build|2665|info|> Build time (Java emits total): 00:01:07.595656
2016-10-06T21:02:39.134719Z|cov-build|2665|info|>
2016-10-06T21:02:39.134719Z|cov-build|2665|info|>
2016-10-06T21:02:39.134763Z|cov-build|2665|info|> Emitted 30 Java compilation units (100%) successfully
2016-10-06T21:02:39.134763Z|cov-build|2665|info|> [WARNING] Recoverable errors were encountered during 1 of these Java compilation units.
2016-10-06T21:02:39.134763Z|cov-build|2665|info|>
2016-10-06T21:02:39.134763Z|cov-build|2665|info|> 30 Java compilation units (100%) are ready for analysis
2016-10-06T21:02:39.134763Z|cov-build|2665|info|> For more details, please look at:
2016-10-06T21:02:39.134763Z|cov-build|2665|info|> /home/travis/build/Edvinas01/chat-rooms/server/cov-int/build-log.txt
However after this finishes, I do not see any submitted builds or changes in projects Coverity dashboard. The project status stays on pending.
I've followed this guide and setup my .travis.yml file like this:
language: java
jdk:
- oraclejdk8
before_script:
- cd server
- chmod +x gradlew
script:
# Run tests when not on coverity branch.
- if [ ${COVERITY_SCAN_BRANCH} != 1 ]; then
./gradlew check;
fi
cache:
directories:
- ~/.gradle
after_success:
# Upload coveralls when not on coverity branch.
- if [ ${COVERITY_SCAN_BRANCH} != 1 ]; then
./gradlew cobertura coveralls;
else
cat cov-int/build-log.txt;
fi
notifications:
email:
on_success: change
env:
matrix:
- TERM=dumb
global:
# COVERITY_SCAN_TOKEN
- secure: "<TOKEN>"
before_install:
- echo -n | openssl s_client -connect scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
addons:
coverity_scan:
project:
name: "Edvinas01/chat-rooms"
description: "Build submitted via Travis CI"
notification_email: "<EMAIL>"
build_command_prepend: "./gradlew clean"
build_command: "./gradlew build"
branch_pattern: coverity_scan
Do I have to specify some additional configuration so that my Coverity builds get published?

Got some time and created a virtual machine with java and the coverity analysis tool. After pulling my project and running the tool I noticed this in the logs:
[WARNING] No files were emitted. This may be due to a problem with your configuration
or because no files were actually compiled by your build command.
After fiddling quite a bit and looking at other projects, I found out that this was due to Gradle version. My project was using 3.0 so I downgraded to 2.14.1 and it finally seems to be working.

For what is worth, there is no issue with using Coverity with any Gradle version, as long as you make sure you are not using the daemon (just to be sure you may specify --no-daemon on the command line).
That said, there are a number of other easy to miss gotchas, resulting in not-quite-obvious error messages.
For useful background, see Caleb's answer here:
Can't get Coverity Scan to work (Java/Kotlin/Gradle 3.3 on Windows and Travis)
For working example, you may refer to this project:
https://github.com/ddimtirov/nuggets

class: Configuration not found on when simulating Hadoop YARN SLS

I am trying to simulate Hadoop YARN SLS (Scheduling Load Simulator) with the sources given in Hadoop's GitHub and the SLS source files are located in [REF-1].
Here the step I have done :
Using VMWARE as the Host.
Using Ubuntu 14.04
Installing Hadoop v 2.6.0 [REF-2]
User : hduser | group : hadoop
Installing any needed packages (e.g. maven)
Get the clonning file of Hadoop's GitHub [REF-1]
Syntax : git clone https://git.apache.org/hadoop.git
Result : hduser#ubuntu:~/hadoop$
I made the changes inside directory hduser#ubuntu:~/hadoop/hadoop-tools$
FYI : I used the codes from MaxinetSLS [REF-3] as the way I compile the source files. The SLS source files can be downloaded by using this syntax in Linux : git clone https://github.com/wette/netSLS.git. By default, I can run this program with no error. The SLS Simulator can work perfectly.
From MaxiNetSLS's source files, I copied this files below into my work in hduser#ubuntu:~/hadoop/hadoop-tools$ :
netSLS/generator > hduser#ubuntu:~/hadoop/hadoop-tools$
netSLS/html > hduser#ubuntu:~/hadoop/hadoop-tools$
netSLS/sls.sh > hduser#ubuntu:~/hadoop/hadoop-tools$
netSLS/sls/hadoop/ > hduser#ubuntu:~/hadoop/hadoop-tools/hadoop-sls$
Then, I modified some files as follows.
netSLS/sls.sh
#!/usr/bin/env bash
function print_usage {
echo -e "usage: sls.sh TraceFile"
echo -e
echo -e "Starts SLS with the given trace file."
}
if [[ -z $1 ]]; then
print_usage
exit 1
fi
TRACE_FILE=$(realpath $1)
if [[ ! -f ${TRACE_FILE} ]]; then
echo "File not found: ${TRACE_FILE}"
print_usage
exit 1
fi
cd hadoop-sls
OUTPUT_DIRECTORY="/tmp/sls"
mkdir -p ${OUTPUT_DIRECTORY}
ARGS="-inputsls ${TRACE_FILE}"
ARGS+=" -output ${OUTPUT_DIRECTORY}"
ARGS+=" -printsimulation"
mvn exec:java -Dexec.args="${ARGS}"
hduser#ubuntu:~/hadoop/hadoop-tools/hadoop-sls/pom.xml$
[REF-4]
hduser#ubuntu:~/hadoop/hadoop-tools$ nano hadoop-sls/hadoop/etc/hadoop/sls-runner.xml
[REF-5]
Next step, I try to :
Compile the script using hduser#ubuntu:~/hadoop/hadoop-tools/hadoop-sls$ mvn compile
Compiled with no error (mvn_compile_perfect.jpg).
Run the program using hduser#ubuntu:~/hadoop/hadoop-tools$ ./sls.sh generator/small.json
Got the error here (error_json_compile.jpg). :(
Until now, I have went through some information related with similar problems I faced [REF-6] and tried it, but I still get the same problem. I guess I think the problem is in the ~/hadoop/hadoop-tools/hadoop-sls/pom.xml I mistakenly modified. I have lack of knowledge with Linux Environment. :(
References : http://1drv.ms/21zcJIH (txt file)
*Cannot post more than 2 links in my post. :(

Open file in Eclipse through URL handler

In my project, I have a special JSP which displays the exception stacktrace in case of Exceptions.
Is there a way to use an URL handler or something else which would have Eclipse open a file? Maybe with xdg-open?
I use Eclipse 4.3 on Kubuntu Linux.

I've ended up with this solution:
Edit xdebug.ini (it should be somewhere like /etc/php/7.0/mods-available/xdebug.ini), add:
xdebug.file_link_format="xdebug://%f(%l)"
Restart your server or php-fpm. For Apache on Ubuntu use sudo service apache2 restart.
Create eclipse-launch.sh. It is intended to parse URL and pass a file to Eclipse. You can name it as you want and put it anywhere you want, I've placed it in the eclise directory. Be sure to replace /home/user with your actual home directory and path="..." with actual eclipse path:
#! /bin/bash
arg=$1
path="/home/user/eclipse/eclipse-neon/"
# file name directly followed by a line number in parenthesis
regex="//([^(]*)\(([0-9]+)\)"
if [[ $arg =~ $regex ]]
then
file=${BASH_REMATCH[1]}
line=${BASH_REMATCH[2]}
$path/eclipse --launcher.openFile "$file"+"$line"
else
msg="Unsupported URL: $arg"
zenity --info --text="$msg"
# alternatives:
# notify-send "$msg" # another notification program
# $path/eclipse # just run eclipse
fi
Read more about Eclipse command line options here: http://help.eclipse.org/mars/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/product_open_file.htm
Give the file executable permissions: chmod +a eclipse-launch.sh
Create xdebug.desktop at ~/.local/share/applications/. It will be used by xdg-open (Chrome uses xdg-open by default).
[Desktop Entry]
Comment=
Exec=/home/user/eclipse/eclipse-neon/eclipse-launch.sh "%u"
Icon=/home/user/eclipse/eclipse-neon/eclipse/icon.xpm
Name=Eclipse xdebug Launch
NoDisplay=false
StartupNotify=true
Terminal=0
TerminalOptions=
Type=Application
MimeType=x-scheme-handler/xdebug;
Run xdg-mime default xdebug.desktop x-scheme-handler/xdebug. This should add an entry to ~.local/share/applications/mimeapps.list to [Default Applications] section. The entry itself should look like x-scheme-handler/xdebug=xdebug.desktop
For Firefox follow instructions from here: https://xdebug.org/docs/all_settings#file_link_format
Open about:config
Add a new boolean setting network.protocol-handler.expose.xdebug and set it to false
The first time you click on xdebug:/// link Firefox will prompt you to select an application to run, point to the created eclipse-launch.sh file.

Automatically stop "tail -f catalina.out" when exception found or server is running

I have set up an automated deployment script (in shell script) for my web application.
It uses java, tomcat, maven and a postgres database.
The deployment script does this:
builds the deployable application from source repository
stops tomcat
applies database migration patches
deploys the war files in tomcat
starts tomcat (by invoking $TOMCAT_HOME/bin/startup.sh)
exits with a success message
It's all working and it's pretty neat - but it needs a little improvement.
You see, even though it exits with a success message, sometimes the deploy was not successful because the web application did not start correctly.
I would like to refactor steps 5 and 6 so that after bring up the tomcat server, the deployment script would "tail -f" in the catalina.out file, looking either for a "server started successfully" message or an exception stack trace.
The tail -f output up to that point should be part of the output of the deployment script, and step 6 would "exit 0" or "exit 1" accordingly.
I know that should be possible, if not in shell script, maybe with python.
The problem is I'm a java specialist - and by specialist I mean I suck at everything else :-)
Help please? :-)

Maybe something like this?
tmp=$(mktemp -t catalina.XXXXXXX) || exit 136
trap 'rm "$tmp"' 0
trap 'exit 255' 2 15
tail -n 200 catalina.out >"$tmp"
if grep -q error "$tmp"; then
cat "$tmp"
exit 1
fi
exit 0
On the other hand, if startup.sh were competently coded, you could just
if startup.sh; then
tail -f catalina.out
else
exit $?
fi
which can be shortened to
startup.sh || exit $?
tail -f catalina.out

As an alternative, you might want to take a look at the Apache Tomcat Manager application. It supports, amongst other things:
Deploying applications remotely, and from local paths
Listing currently deployed applications
Reloading existing applications
Starting an existing application
Stopping an existing application
Undeploying an existing application
The manager provides a web interface that can be called via curl, and which returns simple, parseable messages to indicate the status of the invoked command. Management functions can also be invoked via JMX, or Ant scripts. All in all, a very handy tool.

I ended up implementing a solution using Python's subprocess.Popen, as suggested by #snies.
Here's what it looks like:
waitForIt.py
#! /usr/bin/env python
import subprocess
import sys
def main(argv):
filename = argv[1]
match=argv[2]
p = subprocess.Popen(['tail', '-n', '0', '-f', filename], stdout=subprocess.PIPE)
while True :
line = p.stdout.readline()
print line ,
if match in line :
break
p.terminate()
if __name__ == "__main__":
main(sys.argv)
tailUntil.sh
#!/bin/bash
set -e
filename=$1
match=$2
thisdir=$(dirname $0)
python $thisdir/waitForIt.py "$filename" "$match"
and then
startTomcat.sh
${TOMCAT_HOME}/bin/startup.sh
logDeploy.sh "Agora vamos dar um tail no catalina.out..."
util_tailUntil.sh "$TOMCAT_HOME/logs/catalina.out" 'INFO: Server startup in '
It doesn't do what I originally intended (it still exits with return code 0 even when there is a stacktrace - but that could be changed with a little bit more of Python magic),
but all of tomcat's initialization log is part of the automated deploy out (and easily viewable on Jenkins' deploy job) -
so that's good enough.