Develop Reference

Cant deploy on google cloud app engine using Bitbucket pipeline

I have a java web application deployed in app engine and the source code is in Bitbucket under master branch,
And I heard about bitbucket pipelines I found it helpful as a fast way of auto deploying
My master branch having this list of 4 projects:
master --
|- project1
|- project2
|- project3
|- project4
|- bitbucket-pipelines.yml
And I followed exactly what is written in this link to provide the pipeline functionality:
https://confluence.atlassian.com/bitbucket/deploy-to-google-cloud-900820342.html
and here is my bitbucket-pipelines.yml content and its located directly under my master branch
image: maven:3.3.9
pipelines:
branches:
master:
- step:
caches:
- maven
script:
# Downloading the Google Cloud SDK
- curl -o /tmp/google-cloud-sdk.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-155.0.0-linux-x86_64.tar.gz
- tar -xvf /tmp/google-cloud-sdk.tar.gz -C /tmp/
- /tmp/google-cloud-sdk/install.sh -q
- source /tmp/google-cloud-sdk/path.bash.inc
# Authenticating with the service account key file
- echo $GOOGLE_CLIENT_SECRET | base64 --decode --ignore-garbage > ./gcloud-api-key.json
- gcloud config set project $CLOUDSDK_CORE_PROJECT
- gcloud components install app-engine-java
- gcloud auth activate-service-account --key-file client-secret.json
- cd project1
- mvn clean install package
- 'mvn appengine:update'
CLOUDSDK_CORE_PROJECT : is a pipeline variable contains the project ID
GOOGLE_CLIENT_SECRET : is a pipeline variable contains the base64 encoded service account json file as explained in the attached link
and here is my app engine plugin in the pom.xml
<plugin>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-maven-plugin</artifactId>
<version>${appengine.target.version}</version>
<configuration>
<enableJarClasses>false</enableJarClasses>
<oauth2>false</oauth2>
</configuration>
</plugin>
after I run my pipelines I got this error at executing the line of "mvn appengine:update"
lease visit https://developers.google.com/appengine/downloads for the latest SDK.
********************************************************
The following URL can be used to authenticate:
https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=550516889912.apps.googleusercontent.com&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=https://www.googleapis.com/auth/appengine.admin%20https://www.googleapis.com/auth/cloud-platform
Attempting to open it in your browser now.
Unable to open browser. Please open the URL above and copy the resulting code.
Please enter code: Encountered a problem: No line found
Please see the logs [/tmp/appcfg3177766291803906341.log] for further information.
and then the pipeline result is failed, I looked for this error for 2 days with no hope, I hope come one here help me out
Thanks in advance!

I fixed it, it turned out that the docs in bitbucket is misleading, here is the correct pipeline script, you just need to put this 3 lines of code to build and deploy to google cloud right after:
- mvn install package
- echo $GOOGLE_CLIENT_SECRET > /tmp/client-secret.json
- mvn appengine:update -Dappengine.additionalParams="--service_account_json_key_file=/tmp/client-secret.json"
$GOOGLE_CLIENT_SECRET is an environmental variable having the service account json of App Engine default service account or you can create a new one having project editor privileges

It helped me pass the authenticate error but now I see 403 in my logs. Surprisingly, the version is still getting pushed to app-engine but 0% traffic.
Beginning interaction for module default...
0% Created staging directory at: '/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg16663468200304338426.tmp'
5% Scanning for jsp files.
8% Generated git repository information file.
20% Scanning files on local disk.
25% Initiating update.
28% Cloning 34 application files.
40% Uploading 3 files.
52% Uploaded 1 files.
61% Uploaded 2 files.
68% Uploaded 3 files.
73% Sending batch containing 3 file(s) totaling 41KB.
77% Initializing precompilation...
90% Deploying new version.
95% Closing update: new version is ready to start serving.
98% Uploading index definitions.
Feb. 19, 2018 1:21:24 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #0
Feb. 19, 2018 1:21:25 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #1
Feb. 19, 2018 1:21:25 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #2
Feb. 19, 2018 1:21:25 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #3
Error Details:
2018-02-19 01:20:57.438:INFO::main: Logging initialized #378ms
2018-02-19 01:20:57.575:INFO:oejs.Server:main: jetty-9.3.18.v20170406
2018-02-19 01:20:58.829:INFO:oeja.AnnotationConfiguration:main: Scanning elapsed time=711ms
2018-02-19 01:20:58.843:INFO:oejq.QuickStartDescriptorGenerator:main: Quickstart generating
2018-02-19 01:20:58.859:INFO:oejsh.ContextHandler:main: Started o.e.j.q.QuickStartWebApp#2aceadd4{/,file:///private/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg16663468200304338426.tmp/,AVAILABLE}
2018-02-19 01:20:58.861:INFO:oejs.Server:main: Started #1808ms
2018-02-19 01:20:58.863:INFO:oejsh.ContextHandler:main: Stopped o.e.j.q.QuickStartWebApp#2aceadd4{/,file:///private/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg16663468200304338426.tmp/,UNAVAILABLE}

can't create java application in openshift

I'm developing a web application using Struts 2, Hibernate and etc and not using Maven. I put the needed jars in app.
I put my app in git (https://github.com/vahidhiv/vaphap) and want to use OpenShift to test it but I face this error.
What should I do?
The initial build for the application failed: Shell command '/sbin/runuser -s /bin/sh 55966353e0b8cdebf9000040 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c2,c481' /bin/sh -c \"gear postreceive --init >> /tmp/initial-build.log 2>&1\""' returned an error. rc=255 .Last 10 kB of build output: The jbossews cartridge is already stopped Repairing links for 1 deployments Building git ref 'master', commit 6e5a635 Skipping Maven build due to absence of pom.xml Preparing build for deployment Deployment id is 3ec7fc2a Activating deployment Starting jbossews cartridge jbossews process failed to start ------------------------- Git Post-Receive Result: failure Activation status: failure Activation failed for the following gears: 55966353e0b8cdebf9000040 (Error activating gear: CLIENT_ERROR: Failed to execute: 'control start' for /var/lib/openshift/55966353e0b8cdebf9000040/jbossews # # ) Deployment completed with status: failure postreceive failed

Jenkins: Duplicate building of jars in a single build

I am using Jenkins 1.596 along with Maven 3.1.1. I have multiple branches in my repository and I am using following goals for build:
'-T 5 clean install'
When I start the build, it shows following log:
Started by user anonymous
Building in workspace /var/lib/jenkins/jobs/<my_job>/workspace
> git rev-parse --is-inside-work-tree # timeout=10
....
[workspace] $ /usr/local/apache-maven/apache-maven-3.1.1/bin/mvn -s /var/lib/jenkins/.m2/settings.xml -T 5 clean install
[INFO] Scanning for projects...
It then shows the projects and Reactor build order. It executes the tests successfully and builds all the jars. Following Success message gets displayed after completion of build:
BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:39.752s (Wall Clock)
[INFO] Finished at: Wed Apr 15 00:02:57 BST 2015
[INFO] Final Memory: 104M/406M
Now, the issue is, just after this success message, it repeats the whole build again. It starts scanning all the poms, downloads jars, executes tests and again, I can see the success message. Following is the starting trace of unwanted build:
Parsing POMs
[workspace] $ java -Xmx1024M -cp /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-agent-1.6.jar:/usr/local/apache-maven/apache-maven-3.1.1/boot/plexus-classworlds-2.5.1.jar:/usr/local/apache-maven/apache-maven-3.1.1/conf/logging jenkins.maven3.agent.Maven31Main /usr/local/apache-maven/apache-maven-3.1.1 /var/cache/jenkins/war/WEB-INF/lib/remoting-2.49.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-interceptor-1.6.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-commons-1.6.jar 54000
<===[JENKINS REMOTING CAPACITY]===>channel started
Due to this, the whole build takes twice as much time. I have disabled archiving but that doesn't seem to have affected this. Can anyone please let me know what am I doing wrong here?
Thanks in advance.

I finally managed to figure out the cause of this. Apparently, the 'Pre Steps' was enabled in jenkins which was doing mvn clean install and then the actual build was configures to start which in turn, was doing the same thing :)
I have removed the pre step and it is now building in half amount of time compared to earlier builds.
Hope this helps someone :)

Why does system downloads Google App Engine Java SDK from Maven everytime in a new workspace?

Why does system retrieve Google App Engine Java SDK from Maven everytime I run mvn appengine:devserver in new project? This process takes a lot of time because of large size. Moreover I have already downloaded Google App Engine Java SDK. I have added the <sdk>/bin
in PATH environment variable.
Following are the messages in my terminal.
[INFO]
[INFO] <<< appengine-maven-plugin:1.8.7:devserver (default-cli) # project-ear <<<
[INFO]
[INFO] --- appengine-maven-plugin:1.8.7:devserver (default-cli) # project-ear ---
[INFO]
[INFO] Google App Engine Java SDK - Running Development Server
[INFO]
[INFO] Retrieving Google App Engine Java SDK from Maven
Downloading: http://repo.maven.apache.org/maven2/com/google/appengine/appengine-java-sdk/1.8.7/appengine-java-sdk-1.8.7.zip

Before starting the server, pom.xml in the main application directory should be modified so that inside the element points to the installed App Engine SDK version.
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<appengine.target.version>1.9.5</appengine.target.version>
</properties>
In the above code, version is to be mentioned in the <appengine.target.version> tag.

If you have already downloaded the SDK zip file install it to maven as following.
Change file name and version as required.
mvn install:install-file -Dfile=appengine-java-sdk-1.9.36.zip -DgroupId=com.google.appengine
-DartifactId=appengine-api-1.0-sdk -Dversion=1.9.36 -Dpackaging=zip

Sinatra Jruby Heroku - jruby: No such file or directory -- trinidad (LoadError)

I'm trying to get this application running: github.com/Soliah/sinatra-jruby-heroku.git
One update was necessary according to this release note from Heroku:
http://devcenter.heroku.com/articles/release-note-java-2011-09-29
The build goes fine, without any errors. Bellow some log parts:
[INFO] --- jruby-rake-plugin:1.6.3:jruby (install-bundler) # jruby-heroku ---
[INFO] Successfully installed bundler-1.0.21
[INFO] 1 gem installed
[INFO]
[INFO] --- jruby-rake-plugin:1.6.3:jruby (bundle-install) # jruby-heroku ---
[INFO] Fetching source index for http://rubygems.org/
[INFO] Installing jruby-rack (1.0.10)
[INFO] Installing rack (1.3.2)
[INFO] Installing tilt (1.3.3)
[INFO] Installing sinatra (1.2.6)
[INFO] Installing trinidad_jars (1.0.1)
[INFO] Installing trinidad (1.2.3)
[INFO] Using bundler (1.0.21)
[INFO] Your bundle is complete! It was installed into ./vendor/bundle
(...)
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33.408s
[INFO] Finished at: Tue Jan 31 10:58:03 UTC 2012
[INFO] Final Memory: 9M/490M
[INFO] ------------------------------------------------------------------------
-----> Discovering process types
Procfile declares types -> jruby, web
-----> Compiled slug size is 18.6MB
-----> Launching... done, v5
http://jrubyandjava.herokuapp.com deployed to Heroku
But when I access the deployed application. An application error occurs.
Here is the log, with the error:
$ heroku logs
2012-01-31T10:57:21+00:00 heroku[slugc]: Slug compilation started
2012-01-31T10:58:13+00:00 heroku[web.1]: State changed from created to starting
2012-01-31T10:58:19+00:00 heroku[web.1]: Starting process with command `sh script/jruby -S trinidad -p 52233`
2012-01-31T10:58:20+00:00 app[web.1]: Classpath is: :/app/etc:/app/target/dependency/jruby-complete.jar
2012-01-31T10:58:21+00:00 app[web.1]: jruby: No such file or directory -- trinidad (LoadError)
2012-01-31T10:58:23+00:00 heroku[web.1]: State changed from starting to crashed
2012-01-31T10:58:23+00:00 heroku[web.1]: State changed from crashed to created
2012-01-31T10:58:23+00:00 heroku[web.1]: State changed from created to starting
2012-01-31T10:58:23+00:00 heroku[web.1]: Process exited
2012-01-31T10:58:28+00:00 heroku[web.1]: Starting process with command `sh script/jruby -S trinidad -p 26224`
2012-01-31T10:58:28+00:00 app[web.1]: Classpath is: :/app/etc:/app/target/dependency/jruby-complete.jar
2012-01-31T10:58:31+00:00 app[web.1]: jruby: No such file or directory -- trinidad (LoadError)
2012-01-31T10:58:32+00:00 heroku[web.1]: State changed from starting to crashed
2012-01-31T10:58:33+00:00 heroku[web.1]: Process exited
2012-01-31T10:58:33+00:00 heroku[router]: Error H10 (App crashed) -> GET jrubyandjava.herokuapp.com/ dyno= queue= wait= service= status=503 bytes=
It seems that JRuby is not finding the gems.
But I've tried all kinds of configurations (in script/jruby, heroku config:add, Procfile, etc.) and no one worked.
One more thing: this is the actual heroku config output (stack cedar).
$ heroku config
DATABASE_URL => postgres://kfgubrhars:mYBlJ2YKewqG7LwmPqVI#ec2-50-17-200-129.compute-1.amazonaws.com/kfgubrhars
JAVA_OPTS => -Xmx384m -Xss512k -XX:+UseCompressedOops
MAVEN_OPTS => -Xmx384m -Xss512k -XX:+UseCompressedOops
PATH => /usr/local/bin:/usr/bin:/bin
SHARED_DATABASE_URL => postgres://kfgubrhars:mYBlJ2YKewqG7LwmPqVI#ec2-50-17-200-129.compute-1.amazonaws.com/kfgubrhars
Here is the updated repository: https://github.com/tomasmuller/sinatra-jruby-heroku
Thank's in advance!

Ok! I found the solution. Here are the steps:
Adjust the GEM_HOME, in script/jruby to:
GEM_HOME="$APPDIR"/vendor/bundle
Created the script/bundle, with ENV['GEM_HOME'] and ENV['GEM_PATH'] pointing to 'vendor/bundle' dir.
Adjusted the executions of jruby-rake-plugin, in pom.xml:
install-bundler: <args>-S gem install bundler --no-ri --no-rdoc --install-dir vendor/bundle</args>
bundle-install: <args>script/bundle install --without development:test</args>

There's been a recent change to where the gems get installed. The launcher script/jruby expects the gems to be in .gems but it looks like that has now changed to vender/bundle.
Try changing the line https://github.com/tomasmuller/sinatra-jruby-heroku/blob/master/script/jruby#L94 to export GEM_HOME="$APPDIR"/vender/bundle instead.
I've been meaning to update my blog post with these changes but just haven't got around to it.

As of Bundler 1.2 you are now able to specify the Ruby implementation and version in your Gemfile. The nice thing about this is that Heroku will understand these settings and prepare the your Heroku application for your environment.
Take this Gemfile for example:
source "https://rubygems.org"
ruby "1.9.3"
gem "rails"
gem "puma"
What's cool about this is that by default Celadon Cedar uses Ruby 1.9.2. However, when you specify ruby "1.9.3" in the Gemfile it'll actually compile Ruby 1.9.3 for your Heroku environment.
Now, if you want to add a different Ruby implementation to your Heroku environment, you can do so like this:
source "https://rubygems.org"
ruby "1.9.3", :engine => "jruby", :engine_version => "1.7.0.preview1"
gem "rails"
gem "puma"
Now it'll install and use JRuby 1.7.0.preview1 in Ruby 1.9 mode for your Heroku application upon deployment. It'll also even define the proper JVM options in the Heroku environment variables.
Best of all is that this comes with the official Heroku buildpack, so there is no need to switch to a 3rd party buildpack to get the JRuby/JVM going on Heroku. Although I haven't gotten it to work yet, this should also work with Rubinius, but I believe it's currently bugged. Either that, or I'm doing it wrong.
This is in my opinion an awesome and scalable feature. Just define the Ruby implementation/version/mode you're using in your Gemfile along with your other dependencies and Heroku will ensure the environment is prepared.
It is no longer necessary to use a workaround or use 3rd party buildpacks using this method. It is also no longer necessary to create a hacky Jemfile. Instead, just do everything as you would normally do with MRI, keep the Gemfile, don't use 3rd party buildpacks, just define the Ruby implementation/runtime in the Gemfile via the ruby method and Heroku should take care of things.