Develop Reference

Importing csv data from Storage to Cloud SQL not working - status always "pending"

I am new to java (I have experience with C# though)
Sadly, I inherited a terrible project (the code is terrible) and what I need to accomplish is to import some csv files into Cloud SQL
So there's a WS which runs this task, apparently the dev followed this guide to import data. But it is not working. Here's the code (Essential parts, actually it is longer and more ugly)
InstancesImportRequest requestBody = new InstancesImportRequest();
ImportContext ic = new ImportContext();
ic.setKind("sql#importContext");
ic.setFileType("csv");
ic.setUri(bucketPath);
ic.setDatabase(CLOUD_SQL_DATABASE);
CsvImportOptions csv = new CsvImportOptions();
csv.setTable(tablename);
List<String> list = new ArrayList<String>();
// here there is some code that populates the list with the columns
csv.setColumns(list);
ic.setCsvImportOptions(csv);
requestBody.setImportContext(ic);
SQLAdmin sqlAdminService = createSqlAdminService();
SQLAdmin.Instances.SQLAdminImport request = sqlAdminService.instances().sqladminImport(project, instance, requestBody);
Operation response = request.execute();
System.out.println("Executed : Going to sleep.>"+response.getStatus());
int c = 1;
while(!response.getStatus().equalsIgnoreCase("Done")){
Thread.sleep(10000);
System.out.println("sleeped enough >"+response.getStatus());
c++;
if(c==50){
System.out.println("timeout?");
break;
}
}
public static SQLAdmin createSqlAdminService() throws IOException, GeneralSecurityException {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
GoogleCredential credential = GoogleCredential.getApplicationDefault();
if (credential.createScopedRequired()) {
credential =
credential.createScoped(Arrays.asList("https://www.googleapis.com/auth/cloud-platform"));
}
return new SQLAdmin.Builder(httpTransport, jsonFactory, credential)
.setApplicationName("Google-SQLAdminSample/0.1")
.build();
}
I am not quite sure how response should be treated, it seems it is an async request. Either way, I always get status Pending; it seems it is not even start to executing.
Of course it ends timing out. What is wrong here, why the requests never starts ? I couldn't find any actual example on the internet about using this java sdk to import files, except the link I gave above

Well, the thing is that the response object is static, so it will always return "Pending" as the initial status since it is a string in the object - it is not actually being updated.
To get the actual status, you have to requested it to google using the sdk. I did something like this (it will be better to use a smaller sleep time, and make it grow as you try more times)
SQLAdmin.Instances.SQLAdminImport request = sqlAdminService.instances().sqladminImport(CLOUD_PROJECT, CLOUD_SQL_INSTANCE, requestBody);
// execution of our import request
Operation response = request.execute();
int tried = 0;
Operation statusOperation;
do {
// sleep one minute
Thread.sleep(60000);
// here we are requesting the status of our operation. Name is actually the unique identifier
Get requestStatus = sqlAdminService.operations().get(CLOUD_PROJECT, response.getName());
statusOperation = requestStatus.execute();
tried++;
System.out.println("status is: " + statusOperation.getStatus());
} while(!statusOperation.getStatus().equalsIgnoreCase("DONE") && tried < 10);
if (!statusOperation.getStatus().equalsIgnoreCase("DONE")) {
throw new Exception("import failed: Timeout");
}

retrieve data inside cookieContainer in java

I'm developing a game using Unity engine which have to send cookie from Client side C# to server side - Java , and I facing this problem (maybe cross platform problem? I'm not sure)
I write a bunch of code in client side like this
private HttpWebRequest request(){
try{
string url = "http://localhost:8080/...";
var request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = 15000;
request.KeepAlive = true ;
request.Method= "GET";
CookieContainer cookieContainer = new CookieContainer();
Cookie Authentication = new Cookie("Session" , "09iubasd");
Authentication.Domain = url;
cookieContainer.Add(Authentication);
request.CookieContainer = cookieContainer;
request.Headers.Add("testting", "hascome");
return request;
}catch(System.Exception ex){
Debug.Log("[Exception]" + ex);
throw ex;
}
}
and The server side is writing in Java Spring. I can't retrieve the Cookie data inside the CookieContainer at server-side. Can anyone give me any suggestion or any solution to solve this problem? Or something similar to the CookieContainer in Java. I have googled but seem no way, If this is a silly question then please teach me. Many thanks.
Vince

I just find out the reason why, my cookie domain set wrong way.
Here the new test Code I just fix. Hope this help who have the same problem in the future ( Of cause it must be great if no one face this silly problem )
private HttpWebRequest request(){
try{
System.Uri uri = new System.Uri("http://localhost:8080/...");
var request = (HttpWebRequest)WebRequest.Create(uri);
request.Timeout = 15000;
request.KeepAlive = true ;
request.Method= "GET";
Cookie Authentication = new Cookie("Session" , "09iubasd");
Authentication.Domain = uri.Host;
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(Authentication);
request.Headers.Add("testting", "hascome");
return request;
}catch(System.Exception ex){
Debug.Log("[Exception]" + ex);
throw ex;
}
}

how to post in user's Streams using google plus api in java

I want to share some information in google plus wall from my application.and I am trying for moment.insert, But getting 400 error . Can somebody help me
#Override
public JSONObject getGooglePlusAddUseractivities(Object token) {
Token accessToken = (Token) token;
OAuthService service = createOAuthServiceForGooglePlus();
OAuthRequest request = new OAuthRequest(Method.POST,"https://www.googleapis.com/plus/v1/people/me/moments/vault");
request.addQuerystringParameter("alt", "json");
service.signRequest(accessToken, request);
JSONObject object=new JSONObject();
try {
object.put("kind","plus#moment");
object.put("type","http://schemas.google.com/AddActivity");
JSONObject obj1=new JSONObject();
obj1.put("kind", "plus#itemScope");
obj1.put("url","https://plus.google.com/me");
obj1.put("description","Sign up now to claim and redeem your credits while shopping! ");
obj1.put("image","http://invite.png");
obj1.put("contentUrl", "www.abcd.com");
obj1.put("thumbnailUrl", "http://logo1_favicon.png");
object.putOpt("target", obj1);;
}catch(Exception ex) {
ex.printStackTrace();
}
request.addPayload(object.toString());
request.addHeader("Content-Type", "application/json");
System.out.println("request : "+request.getBodyContents());
Response response = request.send();
String responseBody = response.getBody();
JSONObject googleJSON = null;
try {
googleJSON = new JSONObject(responseBody);
}
catch (JSONException e) {
System.out.println("can not create JSON Object");
}
getting 400 error ?? anyone can tell me..... where am wrong ..!!`

It isn't clear from the documentation, but you can't provide both the target.url and most other target metadata. This is currently opened as bug 485 in the issue tracking system - please go there and star the issue to make sure they properly prioritize a fix.
If you remove the target.url value and add a target.id value, it should work.
(As an aside, this does not post in the user's stream, but will post an App Activity in their app moment vault. They must manually share the activity if they choose.)

At this time, it is not possible to programmatically write to a user's Stream. As a developer, you have two options:
Write an AppActivity (formerly known as a Moment), which writes information to Google, but not to a Google+ Stream. These activities are visible at plus.google.com/apps, and will be used by Google in additional ways over time.
Create an Interactive Post Share button, which a user must initiate. However, you can pre-fill both the text of the post and up to 10 intended recipients. The user can make changes if they want and then perform the actual share. You can learn more at https://developers.google.com/+/web/share/interactive or by watching this Google+ Developers Live episode: http://www.youtube.com/watch?v=U4Iw28jWtAY.

Posting info from Android to PHP Server

So I am writing an Android application, and part of it is supposed to use GPS to acquire longitude/latitude. And those values are supposed to post to a PHP server I also have running. The function I have posting is as follows:
public void doAll() {
Double locLat, locLon;
try {
locLat = locManager.getLastKnownLocation(locProvider).getLatitude();
locLon = locManager.getLastKnownLocation(locProvider).getLongitude();
} catch (NullPointerException e) {
locLat = -1.0;
locLon = -1.0;
}
try {
HttpClient client = new DefaultHttpClient();
//we put all the parameters in the URL itself
HttpPost request = new HttpPost("http://fatalatour.com/database.php?function=get_all&latitude="+Double.toString(locLat)+"&longitude="+Double.toString(locLon));
ResponseHandler <String> responseHandler = new BasicResponseHandler();
//List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
//nameValuePairs.add(new BasicNameValuePair("latitude", Double.toString(locLat)));
//nameValuePairs.add(new BasicNameValuePair("longitude", Double.toString(locLon)));
//request.setEntity(new UrlEncodedFormEntity(nameValuePairs));
String response = client.execute(request, responseHandler);
Log.i("HOMI_RESPONSE", response.toString());
}
There is other code withing that but I believe that is all that actually affects the posting of locLat and locLon.
For the server end I have:
function get_all() {
$lat = trim($_GET['latitude']);
$lon = trim($_GET['longitude']);
// DEBUG
$f = fopen('debug.log', 'a');
fprintf($f, 'get_all: '.print_r($_REQUEST, true)."\n");
$result = mysql_query("SELECT * FROM homi_table");
$num = 0; //# of homicides
while ($row = mysql_fetch_assoc($result)) {
$d = distance($row['latitude'], $row['longitude'], $lat, $lon, "K");
if ($d < 100) { //number to change for distance filter
$num += 1;
echo "fatality:".$row['slug']."~".strtoupper($row['name']).", Age ".$row['age']."\n".$row['date']."\nhttp://projects.latimes.com/homicide/post/".$row['slug']."~".$row['latitude']."~".$row['longitude'];
// GH: limiting the number of results to 100
if ($num == 100) {
break;
}
}
};
echo $num;
fprintf($f, "get_all: returning: ".$num."\n");
}
Right now the issue seems to be the handling of $lat and $lon. When i hard code in a static number for each of those in the PHP the app works, yet when I leave it variable (as it should be), it fails. Any help would be greatly appreciated!

In Android application you are using POST request, in PHP you read GET parameters. Nothing else come to my mind.

The URL you've constructed looks like an HTTP Get, not a Post. Post would have the data being posted in the body of the request.
If you don't mind the parameters being exposed in the URL, I would simply change HTTPPost to HTTPGet in the Android code. If you don't want that stuff to be in the URL, you'll have to rework the Android post a little bit (something similar to your commented out code I believe) and change the php code to access the $_POST variable instead.

Async urlfetch Http post on App engine using Future

My goal is to rapidly make posts to a server from appengine(java). I am attempting to do this using UrlFetchService.fetchAsync. I have been basing my code after this blog post. I have been able to make the request using the code below, however I get some strange behavior:
private void futureRequests() {
URLFetchService fetcher = URLFetchServiceFactory.getURLFetchService();
URL url = new URL("https://someserver.com");
FetchOptions fetchOptions = FetchOptions.Builder.withDefaults();
fetchOptions.doNotValidateCertificate();
fetchOptions.setDeadline(60D);
ArrayList<Future<HTTPResponse>> asyncResponses = new ArrayList<Future<HTTPResponse>>();
for (int i = 0; i < postDatas.size(); i++) {
HTTPRequest request = new HTTPRequest(url, HTTPMethod.POST, fetchOptions);
request.setPayload(postDatas.get(i).getBytes(UTF8));
HTTPHeader header = new HTTPHeader("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
request.setHeader(header);
header = new HTTPHeader("Content-Length", Integer.toString(postDatas.get(i).getBytes().length));
request.setHeader(header);
header = new HTTPHeader("Authorization", "auth=" + authToken);
request.setHeader(header);
Future<HTTPResponse> responseFuture = fetcher.fetchAsync(request);
asyncResponses.add(responseFuture);
}
for (Future<HTTPResponse> future : asyncResponses) {
HTTPResponse response;
try {
response = future.get();
int responseCode = response.getResponseCode();
resp.getWriter().println("response: " + responseCode);
logger.warning("Response: " + responseCode);
} catch (Exception e) {
}
}
}
The strange behavior is that I get duplicate posts on the server, and according to my appstats page I use 10x-20x more urlFetches than what was added with the code above. Below is my appstats screen:
There are more urlFetch calls that could not fit on the screen. It appears that the requests are still completing in a synchronous fashion(circled items), but there are many urlFetches that appear to go on at the same time. My question is how am I getting all this calls to urlFetch when I only had 14 Future ?? Could the server be giving an error or 503 and urlFetch retrying until it goes through? And how can I be getting 2 posts for each request??
I understand that I could use the task queue to do asyc request, however I am dealing with a relatively low number of request(20-100) and the cold start time of ramping up another instance would probably make this not a good option for my situation. Can anyone explain this behavior or have experience with this?

This was simply a mistake in my code that was causing my app to make more request than I thought..