AppEngine Managed VM with Java and Cloud SQL - java

I want a Java Appengine Managed VM application to connect to a 2nd Gen Cloud SQL Instance. There are discrepancies in the documentation - I can't figure out if this is actually supported by Google or not!
https://cloud.google.com/appengine/docs/managed-vms/java/using-cloud-sql
states:
4: In the console, grant your App Engine application access to the Google Cloud SQL instance.
But I see no way of doing this. In the Cloud SQL management console, under properties of an instance, there is:
Authorized applications: None
and seemingly no way to authorize applications?
Then on this page https://cloud.google.com/sql/docs/dev-access it states:
Java App Engine Applications
Using the Cloud SQL Proxy is not supported for Java.
So you seemingly cant use the Cloud SQL proxy. The only way I have got it all working is to open the SQL port to the world, so that the managed VM instances can connect to it on its public IP address, but that is a horrific solution!
Is there an actual supported way of doing this? Anyone from Google able to answer?

April 2016 Update
We have a new Java library for connecting to Cloud SQL instances from Managed VMs and other environments: https://github.com/GoogleCloudPlatform/cloud-sql-mysql-socket-factory
It's still very new so the usual caveats apply, but we haven't found any issues in our testing.
Old answer:
I think the best option right now is to use the junixsocket library as explained in this post: https://stackoverflow.com/a/34820600
If you are using the maven-war-plugin to package your application, then adding the following two dependencies should be enough:
<dependency>
<groupId>com.kohlschutter.junixsocket</groupId>
<artifactId>junixsocket-mysql</artifactId>
<version>2.0.4</version>
</dependency>
<dependency>
<groupId>com.kohlschutter.junixsocket</groupId>
<artifactId>junixsocket-native-common</artifactId>
<version>2.0.4</version>
</dependency>
For Play Framework, add the following dependencies:
libraryDependencies += "com.kohlschutter.junixsocket" % "junixsocket-mysql" % "2.0.4"
libraryDependencies += "com.kohlschutter.junixsocket" % "junixsocket-native-common" % "2.0.4"
Configure your Play application.conf as follows:
db.default.url="jdbc:mysql:///mydb?socketFactory=org.newsclub.net.mysql.AFUNIXDatabaseSocketFactory&junixsocket.file=/cloudsql/PROJECT_ID:REGION:INSTANCE_NAME"
We hope to offer something in the future that does not require the use of junixsocket or a similar library.
We'll review/fix the documentation at https://cloud.google.com/appengine/docs/managed-vms/java/using-cloud-sql as it has some issues. Thanks for bringing it to our attention.

I did finally get it running with a managed VM with this xml:
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>thmadmin-ben</application>
<version>master</version>
<threadsafe>true</threadsafe>
<vm>true</vm>
<precompilation-enabled>false</precompilation-enabled>
<manual-scaling>
<instances>1</instances>
</manual-scaling>
<beta-settings>
<setting name="cloud_sql_instances" value="xxx-ben:us-east1:yyy"/>
</beta-settings>
</appengine-web-app>
But after following the rabit hole I dont think its going to be easy to swap out the TCP database connector with the socket based one in the framework Im using (play framework).
REALLY would love to be able to define "allowed" AppEngine projects in the Cloud SQL instance settings - without this ability Im going to have to run on AWS...

Related

Spring Boot switch between multiple databases (data-sources)

I have a spring boot REST API application for a project with 2 different setups ( =2 different jars). Both need to be realized, but I have no idea on how to do it or what the best way would be.
Both setups need to have a connection with an online database (on a server; AWS) and a connection with an offline or local database (running on the machine/pc itself). It is the offline or local connection that is different between both setups.
Setup 1:
When the application is started it needs to connect to the online database. When an error occurs or connection to the online database is lost, it needs to connect to the offline/local database.
It is not required that that it will reconnect with the online database after an error occurred or the connection was lost.
Setup 2:
When the application is started it needs to connect to both the online and the offline database. So when the user does a post to the REST API, both the online and the offline database need to be updated (unless an error occurs or the connection to the online database is lost). If the user just does a get request, it is preferred to get the data from the online database, unless an error occurs or the connection to the online database is lost, than it can use the offline database.
It is not required that that it will reconnect with the online database after an error occurred or the connection was lost, but in this setup it would be nice.
Synchronizing the data after connection has been established again isn't required either (but would maybe be a nice feature).
I have seen a similar post where the solution was to use ha-jdbc but it is an old post... Maven just doesn't find the dependency when I try adding it to my pom.xml file.
After some more searching and trying I was able to add ha-jdbc to my pom.xml. What I had to do was add a repository that contained ha-jdbc.
<project>
<repositories>
<repository>
<id>jbossrepository</id>
<name>jbossrepository</name>
<url>https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/</url>
</repository>
</repositories>
<dependency>
<dependency>
<groupId>net.sf.ha-jdbc</groupId>
<artifactId>ha-jdbc</artifactId>
<version>3.0.3</version>
</dependency>
</dependency>
</project>
Yet to see if I can get it to work and if it is what I am looking for...
ha-jdbc isn't worth trying, one error or problem after the other, it's just old java too...
Versions:
- java: 1.8
- org.springframework.boot (spring-boot-starter-parent): 2.1.8.RELEASE
- Database (online & offline): PostgreSQL 11.5
Take a look this example code. There you can find already implemented a working solution.
The logic behind it is EnableJpaRepositories which enables it based on packages (basePackages)

WebServlet annotation not working in production (works in development)

I am using Google App Engine STANDARD (SDK 1.9.62) and have set my appengine-web.xml to use java8 runtime and trying to get a simple servlet example. Works as expected in local dev server but gives me a 404 error in the google cloud. I WANT to use the Google App Engine standard environment (not flex). Is there a yaml entry or xml entry I need to turn on for this to work? How can I be sure that GAE is using the 3.1 servlet spec and not 2.5?
App Engine Standard uses appengine-web.xml, whereas Flexible uses app.yaml.
Now, this Java-docs-sample us using the pom.xml to set dependencies. you can see the versions of the dependencies set there.
In this case, that sample can be deployed with mvn appengine:deploy and the endpoint serving in App Engine would be /requests. i.e https://project-name.appspot.com/requests which should return a "hello, world"
Ok. Mystery solved. I had two versions running in deployment and an original java7, servlet 2.5 api version was handling all traffic. My INTENDED version, which was a java8, servlet 3.x was sitting idle and getting 0% traffic. Once I used the version migration function in google's cloud console, the servlets annotated with servlet 3.x annotations started working.

Start and Stop Cloud SQL via Java mysql admin-api

I'm not able to find a way to simply start and stop a Cloud SQL instance using java mysql admin-api.
I found this official google documentation that explain how to start and stop the Cloud SQL instance via gcloud: https://cloud.google.com/sql/docs/mysql/start-stop-restart-instance But I'm not able to obtain the same things via java using mysql admin-api,
Anybody can help me?
Generally, the Cloud SQL Admin API for Java is used for operations such as the one you are looking for. If you are using Maven, you can add the library to your project adding the following lines of code to the pom.xml configuration file:
<project>
<dependencies>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-sqladmin</artifactId>
<version>v1beta4-rev48-1.23.0</version>
</dependency>
</dependencies>
</project>
EDIT:
As far as I can see in the documentation, the underlying API uses the Instance.Patch method for starting and stopping instances, although I cannot find any specific information about how to do it. However, you can find more relevant information yourself in the Instances:Patch page. I will keep looking for more information and in case I find something relevant, I will post a comment to this answer below.
EDIT 2:
I have been performing some tests using the Google APIs Explorer, using the PROJECT_ID, SQL_INSTANCE_ID and a JSON body such as this one:
{
"settings": {
"activationPolicy": "YOUR_PREFERED_STATE"
}
}
According to the documentation:
The activation policy specifies when the instance is activated; it is
applicable only when the instance state is RUNNABLE. Valid values:
ALWAYS: The instance is on, and remains so even in the absence of
connection requests. NEVER: The instance is off; it is not activated,
even if a connection request arrives. ON_DEMAND: First Generation
instances only. The instance responds to incoming requests, and turns
itself off when not in use. Instances with PER_USE pricing turn off
after 15 minutes of inactivity. Instances with PER_PACKAGE pricing
turn off after 12 hours of inactivity.
I have tried running the API with the NEVER and ALWAYS states, and my Cloud SQL instance stopped and started accordingly. So in your case, and going back to the Admin API for Java, you should be looking at the Settings of your instance, specifically at this method:
public Settings setActivationPolicy(java.lang.String activationPolicy)
Changing the Activation Policy to NEVER or ALWAYS should be what you need here, although you can have a look at the other possible instance states in case they fit your requirements better.

How to store JDBC server, database, username and password in a notesdocument

I am using JDBC from ExtLib in my xpages application to connect to SQL and have a JDBC connection file in the Webcontent/WEB-INF/JDBC/mssql.jdbc
Looks something like this and works great
<jdbc>
<driver>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver>
<url>jdbc:sqlserver://server:1433;databaseName=db</url>
<user>sa</user>
<password></password>
</jdbc>
I have a need to make this more dynamic by storing the servername, database, username and password in a notesdocument, is this possible, if so how?
if this is not possible, Is it possible to update the JDBC file programmatically or use other features in ExtLib for making this possible?
We have done this at our company by copying the setup of the JDBCProvider from within the Extension Library, and modifying it so that instead of searching for configuration in xml documents, it will instead search for configuration in Notes documents in using a pre-defined View and Form.
I've just shared it via the jdbcprovider branch in my fork of the ExtLibX project, with the hope to eventually do a pull request to get it into the official OpenNTF ExtLibX, and maybe finally into the core Extension library project.
I have created an 'alpha' release that you can test out, or you can have a browse in the com.ibm.xsp.extlibx.core plugin to see the source code
Here is a blog post that I have written about the first release and there is a short demo video in there as well
http://camerongregor.com/2017/12/05/jdbcnotesdocuments/

ServiceBusConfiguration.configureWithConnectionString can't work, error "The key 'SharedAccessKeyName' is not valid for this connection string"

I'm using newest Azure java sdk(version 0.6), but I found I have problems when I use service bus configuration "configureWithConnectionString" function.
Here is my code below,
//get config
string connectionString = "Endpoint=sb://testservicebusnamespace.servicebus.windows.net/;SharedAccessKeyName=MySharedAccessKey;SharedAccessKey=<this is secret not show>";
config = new Configuration();
ServiceBusConfiguration.configureWithConnectionString(null, config, connectionString);
//create service
service = ServiceBusService.create(config);
And I get connection String from my service bus namespace portal in Azure. But When I run this code, it throws an exception "The key 'SharedAccessKeyName' is not valid for this connection string".
I don't know what's the problem, because I get the connection from Azure portal, and I've checked its content(the SharedAccessKeyName, SharedAccessKey), they are right.
So could anyone help me? Is it my problem or this SDK needs update(because I've heard portal already uses SAS, but SDK still uses ACS to authenticate)?
Thanks very much.
It looks like Java Azure SDK(0.6.0) still uses ACS to authenticate. See this github issue comments:
the current SDK works with service bus namespace with ACS auth mode, but not new created servicebus namespace which use SAS token auth mode. we will investigate this issue
and
One workaround is to create the service bus namespace via PowerShell.
This seems to enable the ACS auth mode by default. Command is:
New-AzureSBNameSpace -Name MyNameSpace -Location "West Europe"
Take a note of the "DefaultKey" you get after running the command. You
need to use that with the ServiceBusConfiguration and I'm not sure if
that is available via the Azure management portal. Use "owner" as the
authenticationName parameter to the configureWithWrapAuthentication.
I'm adding this answer, because I thought I was doomed. But found a new library.
This dependency (which 0.5.0 was latest version when I wrote this post)......gives you the issue.
<dependency>
<groupId>com.microsoft.windowsazure</groupId>
<artifactId>microsoft-azure-api-servicebus</artifactId>
<version>0.5.0</version>
</dependency>
Microsoft slightly altered the groupid. Here is a dependency that seems much more update to date.
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-servicebus</artifactId>
<version>0.9.7</version>
</dependency>
Maven link here : https://search.maven.org/#artifactdetails%7Ccom.microsoft.azure%7Cazure-servicebus%7C0.9.7%7Cjar
So its an old question. But hopefully this points somebody in the right direction.

Categories

Resources