I hope many people already know about the Dropbox Cloud Service for storing and syncing files across various clients. I am little hesitant using a third party service to store my personal files.
I am trying to build a personal cloud storage using my 2TB Hard Drive. I guess I am looking for pointers on where to start, which APIs to use [preferably JAVA and J2EE], security, risks?
First, I would highly recommend getting over your paranoia; chances are extremely slim that Dropbox employees are going to spend their lunch hour looking at your photos or whatever. Literally millions of people store their stuff on Dropbox and nobody's had their privacy violated.
That said, Dropbox is based on Amazon S3, which (since I assume you trust Amazon just as much as you trust Dropbox) has an open source clone that you can run yourself. Take a look at Eucalyptus. Since you specifically brought up Java, I'll point out that the excellent AWS SDK for Java works just as well on Eucalyptus' S3 stores as on Amazon's.
How is it Cloud storage if it's on your 2TB hard drive? Why do you think your own service will be more secure when accessible through the internet then a service run by people who specialized in doing it. I guess Amazon has probably more experience than anybody else in terms of HTTP accessible file storage (S3).
If you want to make sure nobody can look at your stuff I suggest you look into encrypting it before saving it into S3. That would make it harder to access on the client side, because you always need the right tools to do the encryption stuff but that is the price you'd have to pay.
Related
I am working on a school project and need to calculate the time it will take to travel between two locations preferably with the option to specify the mode of transportation inside of an android app.
I have come across the Google Distance Matrix API (documentation here) which seems to have the features that I am looking for, but it warns...
"If you are building a mobile application, you will need to introduce a proxy server to act as intermediary between your mobile application and the Google Maps API Web Services."
I know that this is for the protection of the API key and want to follow this pretty diligently. I'm wonder, is there another way to get the data I need for the app without setting up a proxy server?
I know I could use a service like Heroku to run the code that accesses the API, but this seems like a difficult option given that this is just a school project. Any other ideas on how to get the travel time for my app?
Setting up a proxy server is the best way to secure your API key, but Google also suggests other security techniques such as obfuscation and pinning.
Having said that, if this is a school project and you won't make your API key public in a production environment, then you can just leave your API key unrestricted (and without a proxy server).
But do make sure that you monitor your API usage on a regular basis and that you regenerate your API key periodically and/or as needed. It's also strongly recommended that you set daily quota limits and budget alerts as per this FAQ so that you don't go over budget regardless.
Hope this helps!
I'm continuing with the Java programming of an Android app. Currently I use a local mysql server controlled with phpmyadmin. Having arrived at a good point, I was starting to look for a good hosting site. However, on the internet they speak mostly of the best for some services like wordpress. For this reason, do you have any advice on mysql and Android databases? I was inclined to start with one for free (I found 000webhost for example) but on the internet those that are free are not recommended. At a low price there are several such as hostinger, keliweb and many others. Should I start directly with a paid one? Thank you so much for the answers
Well, it's a good move to make. I have used iPage for about 5 years now and I've never had any issue with them. I also offer hosting with unlimited mysql database, domain, emails and bandwidth as a re-seller agent for iPage at $75/year with no hidden charges. I'm on Twitter
Well if your project is not large scale then it'll be good to go with 000webhost, it offers quiet nice service for free.
Only problem with 000webhost is that you need to use filezilla for better files uploading.
And if you are really looking for paid hosting then go for 1&1 IONOS.
They'll only charge 12$ for first year which will give you a good idea whether you really need a paid hosting or not.
I'm not sure where to ask this so please suggest if I should move this to another stackexchange site
What technologie(s) are sites such as www.jsfiddle.net using in order to store data in the cloud ? Specifically the ability to store a code snippet and then retrieve it later via a URL. Is there a public generic API available for this functionality ?
I've researched cloud offerings such as https://developers.google.com/appengine/docs/java/googlestorage/ but since jsfiddle is free I doubt it is using such a service ?
Via the jsFiddle about page:
jsFiddle’s hosting is kindly provided by DigitalOcean.
There is no such thing as a free lunch: someone has to pay for storage. In this case, it's paid for by a third party, not by jsFiddle developers.
As to what they are using to store their data, it's unclear (though you can ask the developers: their names and websites are listed on that page), but the choice will basically be one of the following:
run a distributed file system on VMs rented from a cloud provider, or
use a managed solution, such as Google Cloud Storage that you pointed to, with an API
Either way, you will incur some costs.
For the sake of brevity consider a facebook style image content serving app. Users can upload content as well as access content shared by other people. I am looking at best ways of handling this kind of file serving application through Java servlets. There is surprisingly little information available on the topic. I'd appreciate if someone can tell me their personal experiences on a small setup (a few hundred users).
So far I am tempted to use the database as a file system (using mongodb) but the approach seems cumbersome and tedious and will need replicating part of the functionality already provided by OS native filesystems. I don't want to use commercial software or have the bandwidth to write my own like facebook. All I want is to be able to do this through free software on a small server with a RAID or something similar. A solution that scales well to multiple servers would be a plus. The important thing is to serve it through java servlets (I am willing to look into alternatives but they have to be usable through java).
I'd appreciate any help. Any references to first hand experiences would be helpful as well. Thanks.
Guru -
I set up something exactly like this for members of my extended family to share photos. It is a slightly complicated process that includes the following:
1) Sign up for Amazon Web Services, notably their S3 (Simple Storage Service). There is a free storage tier that should cover the amount of users you described.
2) Set up a web application that accepts uploads. I use Uploadify in combination with jQuery and ajax, to upload to a servlet that accepts, scans, logs, and does whatever else I want with the file(s). On the servlet side, I use ESAPI's upload validation mechanism, part of the validation engine, which is just built on top of Commons File Upload, which I have also used by itself.
3) After processing the file(s) appropriately, I use JetS3t as my Java-AmazonS3 API and upload the file to Amazon S3. At that point, users can download or view photos depending on their level of access. The easiest way I have found to do this is to use JetS3t in combination with the Web Application Authentication to create Temporary URL's, which give the user access to the file for a specific amount of time, after which the URL becomes unusable.
A couple of things, if you are not concerned with file processing and trust the people uploading their files completely, you can upload directly to Amazon S3. However, I find it much easier to just upload to my server and do all of my processing, checking, and logging before taking the final step and putting the file on Amazon S3.
If you have any questions on the specifics of any of this, just let me know.
While Owens suggestion is an excellent one, there is another option you can consider - what you are describing is a Content Repository.
Since you have sufficient control of the server to be able to install a (non-commercial) piece of software, you may be interested in the Apache Jackrabbit* Content Repository. It even includes a Java API, so you should be able to control the software (at least as far as adding, and extracting content) from your Servlets.
Actually, if you combine this idea with Owens and expand on it, you could actually host the repository on the Amazon S3 space, and use the free-tier Amazon EC2 instance to host the software itself. (Although, I understand that the free-tier EC2 instance is only free for the first year)
HTH
NB. I'm sure other content repositories exist, but JackRabbit is the only one I've played with (albeit briefly).
I assume someone's built an APK decompiler..... What's the best practice to secure sensitive info (like auth parameters for a backend database)? I suppose some kind of middleware would work but that can't do good things for speed. What's the "right way"?
If you're writing an Andoid app and using AWS, strongly recommend you check out:
https://github.com/apetresc/awskeyserver
Author uses the AWS IAM (Identity and Access Management service) with Google App Engine to successfully protect authentication parameters. I imagine IAM will eventually be included in AWS' Android SDK but until then, this is a great option.
It's difficult to reverse-engineer Dalvik byte code; my understanding is that there's not a simple mapping back to Java byte code, much less to Java source, particularly if it's gone through ProGuard. However, auth parameters are usually data, not code, and that can be snooped for rather more easily. Moreover, someone interested in breaking your credentials has lots of other means of attack, including packet sniffing, that don't require recovering your source code. The comment by Anon is exactly right—don't trust the client.
As to best practices, you can use a public key encryption system, get credentials from the server, etc., to avoid putting sensitive info into the .apk file. Don't trust obfuscation or obscure byte code to keep your secrets. They won't.