I am trying to handle a JSON webhook in a Java app that is running in Google App Engine. Right now I am just attempting to post that data into datastore so I can verify it is there.
Here is my current code:
Key storeKey = KeyFactory.createKey("10k", "static");
resp.setContentType("text/plain");
StringBuilder stringBuilder = new StringBuilder(10000);
Scanner scanner = new Scanner(req.getInputStream());
while (scanner.hasNextLine()) {
stringBuilder.append(scanner.nextLine());
}
String body = stringBuilder.toString();
Date date = new Date();
Entity store = new Entity("10k", storeKey);
store.setProperty("Date", date);
store.setProperty("Body", body);
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
datastore.put(store);
It works for a normal POST. But it does not work for a JSON webhook.
Any suggestions were to go? So far googling for it hasn't taken me very far.
Related
I'm using AWS Kinesis Video Stream service to get my video recordings. So due to the Kinesis Video Stream fragment limitation, it turns out I can only retrieve up to ~30 minutes video at one request. And I was intend to retrieve a 2 hour video.
So I loop the request and get all 4 response into a List of InputStream, then I turn them into SequenceInputStream because I try to chain them all together.
However when I success uploaded them to S3 bucket and try to download from there. It shows me file are corrupted. I researched on SequenceInputStream however it seems that my design was okay.
Furthermore, if I extend my video length, let say I have 24 InputStream, and I chained them all to a single SequenceInputStream, it will encounter the SSL Socket Exception: Connection Reset when I run the readAllBytes operation on the sequence input stream.
Is there any way I can achieve what I want or something wrong in my code to cause this?
Here are my source code:
private String downloadMedia(Request request, JSONObject response, JSONObject metaData, Date startDate, Date endDate) throws Exception {
long duration = endDate.getTime() - startDate.getTime();
long durationInMinutes = TimeUnit.MILLISECONDS.toMinutes(duration);
long intervalsCount = durationInMinutes / 30;
ArrayList<GetClipResult> getClipResults = new ArrayList<>();
for (int i = 0; i < intervalsCount; i++){
Media currentMedia = constructMediaAfterIntervalsBreakdown(metaData, request, startDate, endDate);
String deviceName = metaData.getString("name") + "_" + request.getId();
Stream stream = getStreamByName(name, request.getId());
String endPoint = getDataEndpoint(stream.getStreamName());
GetClipResult clipResult = downloadMedia(currentMediaDto, endPoint, stream.getStreamName());
if(clipResult != null){
getClipResults.add(clipResult);
}
startDate = currentMediaDto.getEndTime();
}
//Get presigned URL from S3 service response
String url = response.getJSONArray("data").getJSONObject(0).getJSONArray("parts").getJSONObject(0).getString("url");
if (getClipResults.size() > 0) {
Vector<InputStream> inputStreams = new Vector<>();
for (GetClipResult clipResult : getClipResults){
InputStream videoStream = clipResult.getPayload();
inputStreams.add(videoStream);
}
Enumeration<InputStream> inputStreamEnumeration = inputStreams.elements();;
SequenceInputStream sequenceInputStream = new SequenceInputStream(inputStreamEnumeration);
if (sequenceInputStream.available() > 0){
sequenceInputStream.readAllBytes();
byte[] bytes = sequenceInputStream.readAllBytes();
String message = uploadFileUsingSecureUrl(url, bytes, metaData);
return message;
}
}
return "failed";
}
Edited: I came across couple package that called Xuggler and FFMPEG, however most of them are getting the video file from disk (which has a path), but for my case there isn't any video file because I do not download them to local, they only existed in the runtime and will upload to S3 later on after concatenated.
Appreciates any help! Thank you!
So in the end I just downloaded the clips, saved it to the disk on runtime, merged them using mp4parser and upload to S3. Afterwards I just deleted those on my disk.
If anyone curious about the code, it is taken from https://github.com/sannies/mp4parser/blob/master/examples/src/main/java/com/googlecode/mp4parser/AppendExample.java
Thank you.
I have a PNR Inquiry app on Google Play. It was working very fine. But recently Indian Railwys added captcha to their PNR Inquiry section and because of this I am not able to pass proper data to the server to get proper response. How to add this captcha in my app in form of an imageview and ask the users to enter captcha details also so that I can send proper data and get proper response.
Indian Railways PNR Inquiry Link
If you check the html code, its actualy pretty bad captcha.
Background of captcha is: http://www.indianrail.gov.in/1.jpg
Those numbers are actualy in input tag:
<input name="lccp_cap_val" value="14167" id="txtCaptcha" type="hidden">
What they are doing is, via javascript, use numbers from that hidden input tag
and put them on that span with "captcha" background.
So basicaly your flow is:
read their html
get "captcha" (lol, funny captcha though) value from input field
when user puts data in your PNR field and presses Get Status
post form field, put PNR in proper value, put captcha in proper value
parse response
Oh yeah, one more thing. You can put any value in hidden input and "captcha"
input, as long as they are the same. They aren't checking it via session or
anything.
EDIT (code sample for submiting form):
To simplify posting form i recommend HttpClient components from Apache:
http://hc.apache.org/downloads.cgi
Lets say you downloaded HttpClient 4.3.1. Include client, core and mime
libraries in your project (copy to libs folder, right click on project,
properties, Java Build Path, Libraries, Add Jars -> add those 3.).
Code example would be:
private static final String FORM_TARGET = "http://www.indianrail.gov.in/cgi_bin/inet_pnstat_cgi.cgi";
private static final String INPUT_PNR = "lccp_pnrno1";
private static final String INPUT_CAPTCHA = "lccp_capinp_val";
private static final String INPUT_CAPTCHA_HIDDEN = "lccp_cap_val";
private void getHtml(String userPnr) {
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addTextBody(INPUT_PNR, userPnr); // users PNR code
builder.addTextBody(INPUT_CAPTCHA, "123456");
builder.addTextBody("submit", "Get Status");
builder.addTextBody(INPUT_CAPTCHA_HIDDEN, "123456"); // values don't
// matter as
// long as they
// are the same
HttpEntity entity = builder.build();
HttpPost httpPost = new HttpPost(FORM_TARGET);
httpPost.setEntity(entity);
HttpClient client = new DefaultHttpClient();
HttpResponse response = null;
String htmlString = "";
try {
response = client.execute(httpPost);
htmlString = convertStreamToString(response.getEntity().getContent());
// now you can parse this string to get data you require.
} catch (Exception letsIgnoreItForNow) {
}
}
private static String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line);
}
} catch (IOException ignoredOnceMore) {
} finally {
try {
is.close();
} catch (IOException manyIgnoredExceptions) {
}
}
return sb.toString();
}
Also, be warned i didn't wrap this in async call, so you will have to do that.
Image from the network can be displayed in android via efficient image loading api's like Picasso/volley or simply image view via async task.
considering all above things as basic build a logic such that you should need a image URL for that captcha if user resets or refresh the captcha it should reload new image via network call requesting the new request implementation, you have to get REST api access to the Indian railway and check in that any image uri available in that (it may be in base64 format )
if REST API is not available you may think of building your own server with this code
RESTful API to check the PNR Status
pnrapi
Update: you don't need to do this complex hacks , just implement Drago's answer !
I'm looking to leverage RackSpace's CloudFiles platform for large object storage (word docs, images, etc). Following some of their guides, I found a useful code snippet, that looks like it should work, but doesn't in my case.
Iterable<Module> modules = ImmutableSet.<Module> of(
new Log4JLoggingModule());
Properties properties = new Properties();
properties.setProperty(LocationConstants.PROPERTY_ZONE, ZONE);
properties.setProperty(LocationConstants.PROPERTY_REGION, "ORD");
CloudFilesClient cloudFilesClient = ContextBuilder.newBuilder(PROVIDER)
.credentials(username, apiKey)
.overrides(properties)
.modules(modules)
.buildApi(CloudFilesClient.class);
The problem is that when this code executes, it tries to log me in the IAD (Virginia) instance of CloudFiles. My organization's goal is to use the ORD (Chicago) instance as primary to be colocated with our cloud and use DFW as a back up environment. The login response results in the IAD instance coming back first, so I'm assuming JClouds is using that. Browsing around, it looks like the ZONE/REGION attributes are ignored for CloudFiles. I was wondering if there is any way to override the code that comes back for authentication to loop through the returned providers and choose which one to login to.
Update:
The accepted answer is mostly good, with some more info available in this snippet:
RestContext<CommonSwiftClient, CommonSwiftAsyncClient> swift = cloudFilesClient.unwrap();
CommonSwiftClient client = swift.getApi();
SwiftObject object = client.newSwiftObject();
object.getInfo().setName(FILENAME + SUFFIX);
object.setPayload("This is my payload."); //input stream.
String id = client.putObject(CONTAINER, object);
System.out.println(id);
SwiftObject obj2 = client.getObject(CONTAINER,FILENAME + SUFFIX);
System.out.println(obj2.getPayload());
We are working on the next version of jclouds (1.7.1) that should include multi-region support for Rackspace Cloud Files and OpenStack Swift. In the meantime you might be able to use this code as a workaround.
private void uploadToRackspaceRegion() {
Iterable<Module> modules = ImmutableSet.<Module> of(new Log4JLoggingModule());
String provider = "swift-keystone"; //Region selection is limited to swift-keystone provider
String identity = "username";
String credential = "password";
String endpoint = "https://identity.api.rackspacecloud.com/v2.0/";
String region = "ORD";
Properties overrides = new Properties();
overrides.setProperty(LocationConstants.PROPERTY_REGION, region);
overrides.setProperty(Constants.PROPERTY_API_VERSION, "2");
BlobStoreContext context = ContextBuilder.newBuilder(provider)
.endpoint(endpoint)
.credentials(identity, credential)
.modules(modules)
.overrides(overrides)
.buildView(BlobStoreContext.class);
RestContext<CommonSwiftClient, CommonSwiftAsyncClient> swift = context.unwrap();
CommonSwiftClient client = swift.getApi();
SwiftObject uploadObject = client.newSwiftObject();
uploadObject.getInfo().setName("test.txt");
uploadObject.setPayload("This is my payload."); //input stream.
String eTag = client.putObject("jclouds", uploadObject);
System.out.println("eTag = " + eTag);
SwiftObject downloadObject = client.getObject("jclouds", "test.txt");
System.out.println("downloadObject = " + downloadObject.getPayload());
context.close();
}
Use swift as you would Cloud Files. Keep in mind that if you need to use Cloud Files CDN stuff, the above won't work for that. Also, know that this way of doing things will eventually be deprecated.
I'm designing a android application that has client and server side. The main database is in server which every android device db is trying to look like. Whenever Android device is online, the app syncs with server database(the main database). Because I can change the datas with adding, deleting or updating in the database in server(Users can't change database in Android device, a.k.a Sqlite db). After synchronization, the Sqlite db will be added, deleted or updated.
There are a lot of images in server database, and I'm sending all of data as json type from PHP.
$datas = array('cats' => $cats, 'news' => $news, 'products' => $products);
$datas_json = (object) array('datas' => $datas);
$json = json_encode($datas_json);
echo $json;
And the app reads with BufferReader.
HttpResponse response = httpclient.execute(httppost);
rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
And afterwards, I'm trying to get all data is coming from server as string. There is a real problem about speed.
public String parseFromBuffer(BufferedReader rd) throws IOException
{
String line = "";
String full = "";
while ((line = rd.readLine()) != null)
{
full += line;
}
return full;
}
Because, If you think just one image may have 100.000 characters(in PHP side, I'm using base64 for images to send java and I'm decoding in java again.), imagine 1000 images! So, sometimes it takes 1-2 min to get 4-5 images with this way. After getting 'full' string(it could have 1.000.000 character!), I use jsonParser to get datas.
public jsonParser(String json_string) throws JSONException
{
json = new JSONObject(json_string);
}
...
public void parseJson() throws JSONException
{
JSONObject datas = json.getJSONObject("datas");
JSONArray cats = datas.getJSONArray("cats");
for(int i = 0; i < cats.length(); i++)
{
JSONObject c = cats.getJSONObject(i);
cat_id.add(c.getInt("cat_id"));
cat_name_tr.add(c.getString("cat_name_tr"));
cat_name_eng.add(c.getString("cat_name_eng"));
cat_upper_id.add(c.getInt("cat_upper_id"));
cat_order.add(c.getInt("cat_order"));
}
...
And finally, I insert, update or delete data which comes from jsonParser() to sqlite db.
In short, I have a problem or I need solution about speed of getting images across server-client and also in parseFromBuffer() method. You can suggest any advice, you can criticize of my way of doing this, may be I should serialize something, may be I should download all images in server to 'res' folder somehow...
Thanks in advance.
I want to share some information in google plus wall from my application.and I am trying for moment.insert, But getting 400 error . Can somebody help me
#Override
public JSONObject getGooglePlusAddUseractivities(Object token) {
Token accessToken = (Token) token;
OAuthService service = createOAuthServiceForGooglePlus();
OAuthRequest request = new OAuthRequest(Method.POST,"https://www.googleapis.com/plus/v1/people/me/moments/vault");
request.addQuerystringParameter("alt", "json");
service.signRequest(accessToken, request);
JSONObject object=new JSONObject();
try {
object.put("kind","plus#moment");
object.put("type","http://schemas.google.com/AddActivity");
JSONObject obj1=new JSONObject();
obj1.put("kind", "plus#itemScope");
obj1.put("url","https://plus.google.com/me");
obj1.put("description","Sign up now to claim and redeem your credits while shopping! ");
obj1.put("image","http://invite.png");
obj1.put("contentUrl", "www.abcd.com");
obj1.put("thumbnailUrl", "http://logo1_favicon.png");
object.putOpt("target", obj1);;
}catch(Exception ex) {
ex.printStackTrace();
}
request.addPayload(object.toString());
request.addHeader("Content-Type", "application/json");
System.out.println("request : "+request.getBodyContents());
Response response = request.send();
String responseBody = response.getBody();
JSONObject googleJSON = null;
try {
googleJSON = new JSONObject(responseBody);
}
catch (JSONException e) {
System.out.println("can not create JSON Object");
}
getting 400 error ?? anyone can tell me..... where am wrong ..!!`
It isn't clear from the documentation, but you can't provide both the target.url and most other target metadata. This is currently opened as bug 485 in the issue tracking system - please go there and star the issue to make sure they properly prioritize a fix.
If you remove the target.url value and add a target.id value, it should work.
(As an aside, this does not post in the user's stream, but will post an App Activity in their app moment vault. They must manually share the activity if they choose.)
At this time, it is not possible to programmatically write to a user's Stream. As a developer, you have two options:
Write an AppActivity (formerly known as a Moment), which writes information to Google, but not to a Google+ Stream. These activities are visible at plus.google.com/apps, and will be used by Google in additional ways over time.
Create an Interactive Post Share button, which a user must initiate. However, you can pre-fill both the text of the post and up to 10 intended recipients. The user can make changes if they want and then perform the actual share. You can learn more at https://developers.google.com/+/web/share/interactive or by watching this Google+ Developers Live episode: http://www.youtube.com/watch?v=U4Iw28jWtAY.