I am building an application where audio data is uploaded to my GAE server, processed, and displayed as a response to an HTTP GET request.
Part of the data I wish to display is in the format of a graph. What I am having a hard time understanding is how to create my response in such a way that I can include graphs.
From what I understand, one approach might be to create the graph using this API:
http://googleappsdeveloper.blogspot.ca/2011/09/visualize-your-data-charts-in-google.html
And then store it as a blob in my datastore. I can then create a JSP to serve the blob as an image? Not sure if I am understanding this correctly. Specifically, I'm not sure about being able to access all of this functionality from GAE, and if I'm doing this in a convoluted way.
I am quite new to GAE and web programming in general, so I greatly appreciate feedback and suggestions on how to do this in the simplest/quickest way. I wouldn't mind links to relevant resources as well.
you have mainly two ways to go:
1) Send in your response only data and let your front end (your website or app) parse them and put them in graph form.
You can write in your response the data to show, and it is quite suggested to give them a structure (so as your front-end can easily interpret and validate the data). Common formats are JSON and XML (they basically can give a custom hierarchical structure to your data,for example you can organize the graph data in columns form)
The way to build a graph depends on the technology you use in your front-end and you can either use a third part library or build your own
2) Create graphs in your web application, store them and allow users to get them via HTTP. Once you have found a way to build a graph image from data you need to store it. GAE gives to you two storage systems, the Blobstore and the Google Cloud Storage.
I think You can save files in the Blobstore only by direct upload via HTTP therefore if you're saving image directly in your GAE web app there's no easy way to use it (you should open an HTTP connection).
The Google cloud storage instead can be accessed by using the dedicated libraries (https://developers.google.com/appengine/docs/java/googlecloudstorageclient/getstarted) that you need to download and add to your project during the developing phase (and activate them) there are tutorials for this (https://developers.google.com/appengine/docs/java/googlecloudstorageclient/).
In order to serve images you can bypass the middle code that should read the image from the GCS and serve it as a response by using the Images service. Once generated a so called "serving URL" associated to a given image, the images service permits to directly access to the image via HTTP (https://developers.google.com/appengine/docs/java/images/).
Finally the first option is interesting because (obviously if you can) it's simpler and lighter for the server side (the one you pay) and you can anyway cache the images to avoid useless computation, the second is maybe more correct in a certain point of view but it is more complex.
Related
How can I use ember js without ember-cli with PHP or java? How can I interact with back end data? Any example will be much appreciated.
The first thing you'll need to understand, is that ember-cli is a command line interface. It acts as a blueprint generator and an asset pipeline and glues all the ember stuff together in a wonderful productive package. It's just standard now, and you wouldn't want to use Ember without CLI. They are basically now the same thing unless you were using Ember since pre 1.13 and can't upgrade.
Ember is a JavaScript framework for front-end/client-side user interface. It doesn't deal with server style data storage like a traditional Apache/PHP. In a PHP setup, each page is rendered on the server side and then delivered to the browser in a complete form. Ember is more like a shell that you fill with data from somewhere else, but also has the ability to manipulate that data in the browsers and persist it back to the data-source.
The data source could really be something as simple as local storage. Maybe a simple game that just stores your userScore in the built-in local storage.
The data source could be something like the WordPress API, where ember pulls in posts or page data to display on the screen. Maybe you even have a form that persists new posts back to the server.
You could use parse or firebase for real-time back-end as a service. Or you could build a server of any sort that was able to generate an API that Ember could consume.
Currently, rails is popular for the server. Also, node frameworks like hapi.js and sails. Elixr and Pheonix are gaining interest and have proven to be a powerful backend team with Ember.
Ember uses ember-data and adapters to take in API data and serialize into a simple set of conventions.
If you have API endpoints coming from PHP or Java, you could use them - however, if you are starting a new project - I wouldn't think that would be an ideal route.
You can follow the Ember guides tutorial to learn more about how Ember works. Good luck!.
I want to build an offline browser-based app using HTML and javascript to collect survey data on Android tablets. The app would consist of some static pages with forms for users to enter data, which would then be stored locally using Web Storage or IndexedDB. However, I also want to build a small native Android app which would grab this data and transfer it to other devices. Is this possible, and if so how would I go about it?
Specifically, I want to understand if and how the native app would access the browser's data store (I can manage the rest). I would prefer to use the Android browser but can use any other if that makes it easier. I have found this blog post which suggests that it might be possible but I would appreciate some pointers as to where the data is stored by the Android browser and how easily it can be accessed by another app.
Unfortunately I don't think the data flow can work the way you want it. In the Chromium WebKit implementation, IDB stores data in levelDB files that you should not be able to access (by design).
So how do we get Java and JavaScript to play nice together? That's a great question! As I see it, the only good way to transform Java data into IDB data is via the client-side.
I've got good IDB chops but my Android experience is non-production. From what I understand of it, here's a proposed solution:
collect data via native application views
write a string to a file in your sandbox with the data stored as a JSON blob or in an .html <script> attached to a JavaScript global
load a webview that can access a local URI like file://android_asset/blah.json and then run some IDB code to bulk insert it into IDB
use your IDB store to drive your web-based views
So the answer to "if and how the native app would access the browser's data store" would be: try the opposite. Architect it to let your browser access your native app data store.
Easiest and most robust way to serialise all your records and load into your app when it first run.
It's possible if you're willing to try a slightly different approach:
Pulling data from an HTML5 app is tricky but pushing data to a native app is easier. Your HTML5 app must have a native container. Does the container API include a way to access native ContentProviders? If not, can you add your own native code to the container to do that? Basically, if you can access ContentProviders then your native app need only implement a ContentProvider with insert privileges (which can be restricted to only your HTML5 app). After an insert, the native app can do whatever it wishes with the data, including broadcasting it to to other devices.
If you insist on pulling data from the HTML5 app, this may only be possible if the native and HTML5 app are actually the same app, and that is only possible if your container allows you to add your own native code. Then you will have direct access to the WebView's storage via the WebStorage class.
I am using facebook login on my site. When I test locally I need to use local.mysite.com, so facebook thinks the request is coming from my site. This works great except when I upload images to blobstore. When uploading images app engine always switches to localhost:888. This makes the browser think cross site scripting is happening and prevents my uploads. How can I force app engine to use local.mysite.com instead of localhost:888
This is the error I am getting:
XMLHttpRequest cannot load http://localhost:8888/_ah/upload/agpidWJwcm9qZWN0chsLEhVfX0Jsb2JVcGxvYWRTZXNzaW9uX18YBQw. Origin http://local.mysite.com:8888 is not allowed by Access-Control-Allow-Origin.
I'm not sure you can actually change that URL.
What you can do though is to use the localhost:8888 for your local tests and create another Facebook application that points to localhost. Afterwards there are two approaches that you can do in order be able to use these two (or possibly even more in the future) Facebook applications in your app.
You can decided based on the requested URL which key to use
Store all the keys in somekind configuration Datatstore that only admins can change them
With the first approach you will have to store somehow all the keys in your code or even worse in the datastore and then decided based on the URL which one to use. This approach is not good and it doesn't scale very well. The second approach is preferable since you don't have to store your keys in the code, it is more secure and it scales much better since you don't need to know up front how many different Facebook applications you have.
You can read the Nick Johnson's answer on how to solve that in Python, but the idea is Java so it shouldn't be that hard.
My project is a computer vision java application which should implement the following :
A web interface through which the form entry+images(for example a student data) will be stored into a database(Mysql) & images into directory common to my java application.
Then the data & images can be retrieved from my java Gui application and I can perform the following operations of image processing through OpenCV.
Actually,I want to run the face detection on images retrieved and discard the false entries(no proper face).
Also the application user/admin can search an image based on text search(By Id) or By another reference image using face recognition.
I am well familiar with Java but the problem is that I need a guidance on how to organise it in a stepwise manner(links appreciated).OpenCv,Php and mySql are really messy.I know doing the openCV stuff within java is real overhead but i really want to do it.But If there is any suggestion to do it elseway please guide me.So any kind of help is a ray of hope for me.
Thanks.
This sounds like a nice application. Here is what I would suggest you:
1- A web interface through which the form entry+images(for example a student data) will be stored into a database(Mysql) & images into directory common to my java application.
You will be able to easily create such application with Php and Mysql, just look for some tutorials on image uploading and php.
2- Then the data & images can be retrieved from my java Gui application and I can perform the following operations of image processing through OpenCV.
I'd recomend you using JavaCV. It is a wrapper over OpenCV which seems to work very well from some tests I have made. You get the flexibility of Java and the performance of C++ through a native bridge.
3- Actually,I want to run the face detection on images retrieved and discard the false entries(no proper face).
You can easily adapt the OpenCV face detection code to JavaCV: http://opencv.willowgarage.com/wiki/FaceDetection
4- Also the application user/admin can search an image based on text search(By Id) or By another reference image using face recognition.
The search by id is easily accomplished through some JPA or even JDBC connection. Now the similar face search is tricky. I'd point you to this link http://www.shervinemami.info/faceRecognition.html about face recognition in OpenCV.
I hope that helps.
Best regards,
Daniel
Here how it can go as far as I can suggest though will need to take care of issue that might not be addressed here.
You will be creating a webservice that will be accessed directly for to upload images. The same service can be used to download the images from your Java application. So make sure you have created a well defined service that takes care of these two distinct operations and are cleanly exposed to the outside world. MySQL will be the back end DB that shall take care of storing either student info and/or images themselves. You can write a service without MySQL and just have the service store images and/or student info locally (at the server end) and later on you can develop code to integrate MySQL. A good and simple link to MySQL and Php
JavaCV Would be the candidate for your java application. You will leverage the power of OpenCV easily. Develop the application and test it against your service. Try just downloading some image from the service first. If that works, go on to Image processing part.
Make sure you undetstand Face Detection and Recognition. These are two different things. From the OpenCV wiki FaceDetection and FaceRecognition
Searching via id/text string would be trivial if you get through step one and understand querying data. Searching via image would me you would store the face recognition artifact (vectors,dimensions,eigen values etc..whatever). Once store on your server, your application will then send off a input image artifact to your server where matching will be done and result returned.
**
To summarize
**
Connecting your webservice and MySQL via connectors such as JDBC. This is very important. No SQL connection, means no DB and a useless service.
Service taking care of student info upload/download.
Image processing. Face artifacts calculation must be common. Same code can run at server and client (makes sense, doesn't it?).
Querying for search either by string or image. Two different types of searches. Client must tell what type of search the server should do.
Requirement is to keep a copy of complete web page at server side same as it is rendered on client browser as past records.These records are revisited.
We are trying to store the html of rendered web page. The html is then rendered using resources like javascript, css and image present at server side. These resources keep on changing. Therefore old records are no longer rendered perfectly.
Is there any other way to solve above? We are also thinking converting it into pdf using IText or apache FOP api but they does not consider javascript effect on page while conversion. Is there any APIs available in java to achieve this?
Till now, no approach working perfectly. Please suggest.
Edit:
In summary,requirement is to create a exact copy of rendered web page at server side to store user activities on that page.
wkhtmltopdf should do this quite nicely for you. It will take a URL, and return a pdf.
code.google.com/p/wkhtmltopdf
Example:
wkhtmltopdf http://www.google.com google.pdf
Depending on just how sophisticated your javascript is, and depending on how faithfully you want to capture what the client saw, you may be undertaking an impossible task.
At a high level, you have the following options:
Keep a copy of everything you send to the client
Get the client to return back exactly whatever it has rendered
Build your system in such a way that you can actually fetch all historical versions of the constituent resources if/when you need to reproduce a browser's view.
You can do #1 using JSP filters etc, but it doesn't address issues like the javascript fetching dynamic html content during rendering on the client.
Getting the client to return what they are seeing (#2) is tricky, and bandwidth intensive.
So I would opt for #3. In order to turn a website that renders dynamic content versioned, you have to do several things. First, all datasources need to versioned too. So any queries would need to specify the version. "Version" can be a timestamp or some generation counter that you maintain. If you are taking this approach, you would also need to ensure that any javascript you feed to the client does not fetch external resources directly. Rather, it should ask for any resources from your system. Your system would in turn fetch the external content (or reuse from a cache).
The answer would depend on the server technology being used to write the HTML. Are you using Java/JSPs or Servlets or some sort of an HTTPResponse object to push the HTML/data to the browser?
If only the CSS/JS/HTML are changing, why don't you just take snapshots of your client-side codebase and store them as website versions?
If other data is involved (like XML/JSON) take a snapshot of those and version that as well. Then the snapshot of the client codebase as mentioned above with the contemporary snapshot of the data should together give you the exact rendering of your website as at that point of time.
A very resource-consuming requirement but...
You haven't written what application server you are using and what framework. If you're generating responces in your own code, you can just store it while generating.
Another possibility is to write a filter, that would wrap servlet's OutputStream and log everything that was written to it, you must just assure your filter is on the top of the hierarchy.
Another, very powerfull, easiest to manage and generic solution, however possibly the most resource-consuming: write transparent proxy server staying between user and application server, that would redirect each call to app server and return exact response, additionally saving each request and response.
If you're storing the html page, why not the references to the js, css, and images too?
I don't know what your implementation is now, but you should create a filesystem with all of the html pages and resources, and create references to the locations in a db. You should be backing up the resources in the filesystem every time you change them!
I use this implementation for an image archive. When a client passes us the url of an image we want to be able to go back and check out exactly what the image was at that time they sent it (since it's a url it can change at any time). I have a script that will download the image as soon as we receive the url, store it in the filesystem, and then store the path to the file in the db along with other various details. This is similar to what you need, just a couple more rows in your table for the js, css, images paths.