I'm not sure where to ask this so please suggest if I should move this to another stackexchange site
What technologie(s) are sites such as www.jsfiddle.net using in order to store data in the cloud ? Specifically the ability to store a code snippet and then retrieve it later via a URL. Is there a public generic API available for this functionality ?
I've researched cloud offerings such as https://developers.google.com/appengine/docs/java/googlestorage/ but since jsfiddle is free I doubt it is using such a service ?
Via the jsFiddle about page:
jsFiddle’s hosting is kindly provided by DigitalOcean.
There is no such thing as a free lunch: someone has to pay for storage. In this case, it's paid for by a third party, not by jsFiddle developers.
As to what they are using to store their data, it's unclear (though you can ask the developers: their names and websites are listed on that page), but the choice will basically be one of the following:
run a distributed file system on VMs rented from a cloud provider, or
use a managed solution, such as Google Cloud Storage that you pointed to, with an API
Either way, you will incur some costs.
Related
I am working on a school project and need to calculate the time it will take to travel between two locations preferably with the option to specify the mode of transportation inside of an android app.
I have come across the Google Distance Matrix API (documentation here) which seems to have the features that I am looking for, but it warns...
"If you are building a mobile application, you will need to introduce a proxy server to act as intermediary between your mobile application and the Google Maps API Web Services."
I know that this is for the protection of the API key and want to follow this pretty diligently. I'm wonder, is there another way to get the data I need for the app without setting up a proxy server?
I know I could use a service like Heroku to run the code that accesses the API, but this seems like a difficult option given that this is just a school project. Any other ideas on how to get the travel time for my app?
Setting up a proxy server is the best way to secure your API key, but Google also suggests other security techniques such as obfuscation and pinning.
Having said that, if this is a school project and you won't make your API key public in a production environment, then you can just leave your API key unrestricted (and without a proxy server).
But do make sure that you monitor your API usage on a regular basis and that you regenerate your API key periodically and/or as needed. It's also strongly recommended that you set daily quota limits and budget alerts as per this FAQ so that you don't go over budget regardless.
Hope this helps!
For the sake of brevity consider a facebook style image content serving app. Users can upload content as well as access content shared by other people. I am looking at best ways of handling this kind of file serving application through Java servlets. There is surprisingly little information available on the topic. I'd appreciate if someone can tell me their personal experiences on a small setup (a few hundred users).
So far I am tempted to use the database as a file system (using mongodb) but the approach seems cumbersome and tedious and will need replicating part of the functionality already provided by OS native filesystems. I don't want to use commercial software or have the bandwidth to write my own like facebook. All I want is to be able to do this through free software on a small server with a RAID or something similar. A solution that scales well to multiple servers would be a plus. The important thing is to serve it through java servlets (I am willing to look into alternatives but they have to be usable through java).
I'd appreciate any help. Any references to first hand experiences would be helpful as well. Thanks.
Guru -
I set up something exactly like this for members of my extended family to share photos. It is a slightly complicated process that includes the following:
1) Sign up for Amazon Web Services, notably their S3 (Simple Storage Service). There is a free storage tier that should cover the amount of users you described.
2) Set up a web application that accepts uploads. I use Uploadify in combination with jQuery and ajax, to upload to a servlet that accepts, scans, logs, and does whatever else I want with the file(s). On the servlet side, I use ESAPI's upload validation mechanism, part of the validation engine, which is just built on top of Commons File Upload, which I have also used by itself.
3) After processing the file(s) appropriately, I use JetS3t as my Java-AmazonS3 API and upload the file to Amazon S3. At that point, users can download or view photos depending on their level of access. The easiest way I have found to do this is to use JetS3t in combination with the Web Application Authentication to create Temporary URL's, which give the user access to the file for a specific amount of time, after which the URL becomes unusable.
A couple of things, if you are not concerned with file processing and trust the people uploading their files completely, you can upload directly to Amazon S3. However, I find it much easier to just upload to my server and do all of my processing, checking, and logging before taking the final step and putting the file on Amazon S3.
If you have any questions on the specifics of any of this, just let me know.
While Owens suggestion is an excellent one, there is another option you can consider - what you are describing is a Content Repository.
Since you have sufficient control of the server to be able to install a (non-commercial) piece of software, you may be interested in the Apache Jackrabbit* Content Repository. It even includes a Java API, so you should be able to control the software (at least as far as adding, and extracting content) from your Servlets.
Actually, if you combine this idea with Owens and expand on it, you could actually host the repository on the Amazon S3 space, and use the free-tier Amazon EC2 instance to host the software itself. (Although, I understand that the free-tier EC2 instance is only free for the first year)
HTH
NB. I'm sure other content repositories exist, but JackRabbit is the only one I've played with (albeit briefly).
I have a GWT based app, which is deployed on Google App Engine for Java. The app uses Google Accounts based authentication. I'm maintaining basic user information such as email id (from Google Accounts), last login date etc. in the GAE datastore. The access to the website is free. Anyone can use it using his/her Google Account.
Going forward, I would like to make it a paid service. However, I have zero experience with setting up and operating an e-commerce website. So my question may be a bit vague. I need some guidance on how to go about it.
Here are some of my requirements (but I'm flexible on the exact implementation):
Offer 2 different types of account - free and premium.
I do not want to maintain any credit card related information in my system. I would also prefer to not maintain an elaborate user database.
When a user first logs in, he/she automatically gets a free account.
User has to "upgrade" to a premium account to access all features of the application.
User has to pay a one-time fee to upgrade.
Given this information, I have the following questions:
Is GAE suited for my requirements?
Which payment gateway (Paypal, Google Checkout etc.) would be most suitable for my requirements?
What level of integration is required between my app and the Payment gateway? I would like to maintain minimal user information in my app. I want to focus on my application development and want to spend minimal effort on user administration.
Would I need to implement a custom authentication mechanism or continue to use Google Accounts or another OpenID based authentication?
What other things do I need to consider?
I'll appreciate any help on this. Thanks.
Generally speaking, there's absolutely no reason why you wouldn't be able to keep the current application and its account management. You can extend your user account with an account type field that stores whether the user is a paying customer or not. If you need to send invoices, also store the users contact information (Paypal will send this to you together with payment receipts)
As far as specific payment providers is concerned. I only have practical hands-on experience with PayPal. I would not use them again for a few reasons:
Their APIs are not very well documented and some of the documentation is wrong (or out-of-date).
If you're a small player, support is mostly via the forums. So basically this means you're on your own.
Some of the APIs have serious gaping holes and missing functionality (for instance, you can create subscriptions, but not cancel them if you're using the standard payments APIs.
Outside of the US and a few lucky countries, the advanced APIs are not available. So you're stuck with implementing a IPN listener servlet, while it would much more preferable to pull the information when needed.
All existing PayPal Java libraries I have found use the Pro features that are not available to most of the users. Because I couldn't find it anywhere else, I have created and open sourced my own IPN servlet but it is very unfinished. If there is a demand for it, I'd be willing to improve it, just let me know.
What this IPN servlet does is listen to incoming PayPal messages. For instance, if a user subscribes, you'll get a message. If a user is billed (for instance on a monthly cycle), you'll get a message. If a user cancels his subscription you'll get a message. Those message allow you to maintain the account type of the user.
If I'd do it again, I would probably use a more advanced and higher level subscription API, such as Spreedly. I've heard some good things about the API and they are pretty affordable. I have no actual experience with Spreedly, so this is not an endorsement.
GAE supports this kind of applications without any particular problem; if your language of choice is Java, I would pick Paypal with this toolkit because Google Checkout Java API does not seem to work very well on GAE.
You would need an Authorization mechanism to check what your users are allowed to do based on their permissions.
Basically you would need the following things:
A flag membership status that indicates if a user is Premium or not; this should be set after the payment notification
An authorization system to check if, reading the membership flag value
, a given Web Handler can be consumed by the current user
Have a look to this great Spring Security tutorial; it covers:
Authentication using Google Accounts.
Setup access-control restrictions based on the roles assigned to users.
If you need subscription based billing with free trials, coupons, plan upgrades/downgrades, Dunning management and a whole lot more then take a look at Recurly.
You can choose to have customers maintain their billing information on pages hosted by Recurly, or use their Transparent POST API to post billing info to their servers from a page in your GAE app - both solutions avoid passing sensitive credit card data through your servers, which makes PCI compliance easy.
The Java API is not fully developed but it's easy to build out for your specific requirements using JAXB. I do not have my code wrapped into an open source project but would be willing to share snippets.
I hope many people already know about the Dropbox Cloud Service for storing and syncing files across various clients. I am little hesitant using a third party service to store my personal files.
I am trying to build a personal cloud storage using my 2TB Hard Drive. I guess I am looking for pointers on where to start, which APIs to use [preferably JAVA and J2EE], security, risks?
First, I would highly recommend getting over your paranoia; chances are extremely slim that Dropbox employees are going to spend their lunch hour looking at your photos or whatever. Literally millions of people store their stuff on Dropbox and nobody's had their privacy violated.
That said, Dropbox is based on Amazon S3, which (since I assume you trust Amazon just as much as you trust Dropbox) has an open source clone that you can run yourself. Take a look at Eucalyptus. Since you specifically brought up Java, I'll point out that the excellent AWS SDK for Java works just as well on Eucalyptus' S3 stores as on Amazon's.
How is it Cloud storage if it's on your 2TB hard drive? Why do you think your own service will be more secure when accessible through the internet then a service run by people who specialized in doing it. I guess Amazon has probably more experience than anybody else in terms of HTTP accessible file storage (S3).
If you want to make sure nobody can look at your stuff I suggest you look into encrypting it before saving it into S3. That would make it harder to access on the client side, because you always need the right tools to do the encryption stuff but that is the price you'd have to pay.
I need to extract data from a Java web application. To be specific I am looking to extract real time stock data from yahoo market tracker. Can anyone please suggest any method?
I'm not sure you can extract the data from Yahoo Market Tracker. Even if you can, you might not be allowed to - I can't see any obvious terms & conditions/licensing. I think (although I could be wrong, anyone got better info?) that you'll need to pay to get access to an API providing near realtime market data.
There is a HTTP-based Yahoo Stock Quote API you could use to get prices, described here. Very simple, returns a comma-separated list of attributes for one or more stock symbols, for example:
http://finance.yahoo.com/d/quotes.csv?s=MSFT&f=snd1l1yr
It might not be realtime enough, but it might be the best you can do for free.
You can use glorious HTTP protocol to do that. Use any language you are comfortable with (Java, C#, VB.NET, python, ruby, php) and crawl the website you are trying to get information from.
I need to extract data from a Java web
application
From your standpoint, the fact that it is a Java Webapp or a PHP-one or static html pages doesn't change anything. It is not because Java is backing the webapp that suddenly you get a "Java-way" to extract the info.
Now in some cases there are APIs provided allowing you to interact with the data present on the website: but once again the fact that the Webapp is a Java one or not bears no importance.