I am writing a multiplayer game where I allow players to attack each other.
Only the attacker must be logged in.
I need to know how many attacks player did in last 6 hours, and I need to know if the defender was attacked during last 1 hour. I don't care about attacks done more than 6 hours ago. Is there any way to implement it better than storing these data in database and deleting "expired" data (older than 6 hours)?
Server is written in java, clients will be Android.
Any ideas / tutorial links or even keywords are appreciated. Also, if you think there is no better solution, please say so :)
For GAE, there is no real alternative, no. Besides using the datastore, GAE offers a memcache system, which, in fact, is designated for storing temporary data. However, as stated in the documentation, "Values can expire from the memcache at any time, and may be expired prior to the expiration deadline set for the value." Therefore, it would be the best to use the datastore.
Related
I need to know what's the best way to store persistent data on google app engine in a situation like mine:
I only need to store a few relatively small amounts of data (a few hashmaps of strings to strings, with no string exceeding over ~300 characters). i'm estimating maybe maximum usage is 5 of these hashmaps and 3000 keys per hash map... The data needs to be around indefinitely, and will be updated once every ~30 minutes. would REALLY prefer not to have pay for service, as it would require a ton of beaurocratic procedures (corporate policy).
I've been trying to wrap my head around how data storage really works on app engine...
can i just continue to use a static HashMap<String, String> to store all the data? this is how i currently have my deployment set up, and it seems to be working just fine for the past couple of hours (no data reset). i dont think i have any backend instances or anything set up.
how long can i expect this data to last if it is in memcache or jcache? the documentation says it's volatile, but if something is used incredibly infrequently, will it just stay forever in cache?
I've been looking at the Datastore options, but i'm so confused on what/if anything will cost money.
can i store it in just like a raw .txt file somehow??
any help would be appreciated. thanks!
You can setup a little table on the DB, so that you have your pairs stored on a non-volatile place. You cannot rely on Memcache if these data should be preserved.
for point #4, you cannot use the filesystem on backend instances.
Hope this helps
I've read this post which was very near my question and I still didn't found what I was looking for.
I'm developing an application that relies on two plain-text files: let's say weekdays.txt and year.txt. One file has most likely (yet to define) seven lines, so it's very small (few bytes), but the other will contain 365 lines (one per each day of the year), which is not very big in bytes (20 Kb tops, my guess), but requires more processing power.
The app is not yet done, so I'll try to be explicit:
So my application will get the current date and the time and will look on weekdays.txt for the line that corresponds to the current day of the week, parse that line's information and store it in memory.
After that the program should read year.txt and look for the line that corresponds to the current date and parse (and store in memory) that line's info.
Then it should do print out all the stored info.
When I say 'parse the info' I mean parsing Strings, something as simple as:
the string "7*1234-568" should be read as:
String ID=7;
int postCode=1234;
int areaCode=568;
The goal here is to create a light (and offline, this is crucial) application for quick use.
As you can see, this is a Developing 101 level application, and my question is: do you think this is too heavy work for any mobile phone? The reason I'm asking this is because I want my app to be functional in the biggest number of today's cellphones possible.
By the way, do you think for this kind of work I should instead be working with a database? I heard people around the forum talking of RMS and some said that it's kind of limited, so I just stayed the same. Anyway the idea of the txt files was to be easiest for the user to update just in case it's necessary...
Thanks in advance!
If your config files are read-only and are not going to change with time, then you could include them inside the jar. You should be able to read them using Class.getResourceAsStream that returns an InputStream. An ASCII file with 366 lines (remember leap years) and 80 cols is around 29KB, so even 10 years old phones will read it without major problems (remember to perform IO in a separate thread though).
If the configuration could change, then you'll probably want to create a WS and have the phones fetch the config over the internet. To provide offline capabilities you could sync with the remote DB periodically and store the info in the device. RMS is record-based, and has a max size (device-dependent), but I think it is ok for your case. The drawback of this approach is that at least a first synchronization should be made, thus phones without a data plan will be left out.
Since one of your requirement is to do it offline, I'd recommend using the RMS. I am not that confident in using files in j2me for such important data (not sure if it's better now) since it can be prone to errors and file corruptions.
If the amount of data you're going to save is as you say, 7 lines for weeks and 365 for years, then no problems with RMS.
Good luck!
We have a java based product which keeps Calculation object in database as blob. During runtime we keep this in memory for fast performance. Now there is another process which updates this Calculation object in database at regular interval. Now, what could be the best strategy to implement so that when this object get updated in database, the cache removes the stored object and fetch it again from database.
I won't prefer any caching framework until it is must to use.
I appreciate response on this.
It is very difficult to give you good answer to your question without any knowledge of your system architecture, design constraints, your IT strategy etc.
Personally I would use Messaging pattern to solve this issue. A few advantages of that pattern are as follows:
Your system components (Calculation process, update process) can be loosely coupled
Depending on implementation of Messaging pattern you can "connect" many Calculation processes (out-scaling) and many update processes (with master-slave approach).
However, implementing Messaging pattern might be very challenging task and I would recommend taking one of the existing frameworks or products.
I hope that will help at least a bit.
I did some work similar to your scenario before, generally there are 2 ways.
One, the cache holder poll the database regularly, fetch the data it needs and keep it in the memory. The data can be stored in a HashMap or some other collections. This approach is simple and easy to implement, no extra framework or library needed. But users will have to endure dirty data from time to time. Besides, polling will cause a lot of pressure on DB if the number of pollers is huge or the query is not fast enough. However, it is generally not a bad one if your requirement for real-time is not that high and the scale of your system is relatively small.
The other approach is that the cache holder subscribes the notification of the data updater and update its data after being notified. It provides better user experience, but this will bring more complexity to your system because you have to get some MS infrastructure, such as JMS, involved. Developing and tuning is more time-consuming.
I know I am quite late resonding this but it might help somebody searching for the same issue.
Here was my problem, I was storing requestPerMinute information in a Hashmap in a Java filter which gets loaded during the start of the application. The problem if somebody updates the DB with new information ,the map doesn't know about this.
Solution: I took one variable updateTime in my Java filter which just stored when was my hashmap last got updated and with every request it checks if the current time is time more than 24 hours , if yes then it updates the hashmap from the database.So every 24 hours it just refreshes the whole hashmap.
Although my usecase was not to update at real time so it fits the use case.
HI I need to find the showtimings in all the theaters of US. I gather that I can get the information of a perticular theater using http://gateway.moviefone.com/movies/pox/closesttheaters.xml?zip=zipcode .. but i dont have the list of zipcodes and theaterid's
can someone help me with that
I don't understand exactly what it is you are doing, but I suspect that you are using an unpublished web API to snarff content from the moviefone services.
Beware.
If that is what you are doing, it is likely to fall foul of the AOL Terms of Service.
I will use this data for a feature in a site (which is going to be plublished soon.Sorry I cannot provide much information abt this site because my client has asked me to keep it confidential).
The fact that you are doing this for a client does not absolve you of legal responsibility for your actions. And, in fact, it potentially exposes your client to legal risk as well. If AOL decide to sue someone over this, you both could be named as defendants.
Another possible outcome is that AOL could use technical means to prevent your systems from snarffing the data.
Can you please tell me any other site or list of sites which provide API for theater showtimes or from where I can snarf data on a weekly basis.
No I can't.
The point is that this data you are snarfing costs someone a significant amount of money to create / assemble / manage. AOL will paying at least part of that bill, one way or another, and they won't take kindly to someone (like your client) freeloading off them and (potentially) taking away their business as well.
I am planning to develop some application like connecting with friends of friends of friends. It may look like as Facebook or Twitter but initially i am planning to implement that to learn more about NOSQL databases.
There are number of database tools in NOSQL. I have gone through many database types like document store, key-value store, column type, graph databases. And finally i come up with two database tools which are cassandra & Neo4J. Is it right to choose any one, if not correct me & provide me some your valuable opinions.
One more thing is the language binding which i choose is JAVA.
My question is,
Which database tool suits for my application?
Awaiting for your valuable opinions. Thanks for spending your valuable time.
Tim, you really should have posted your question separately, rather than as an answer to the OP, which it wasn't.
But to answer, first, go read Ben Black's slides at http://www.slideshare.net/benjaminblack/introduction-to-cassandra-replication-and-consistency.
Done? Okay, now for the specific questions:
"How would differences in [replica] data-state be reconciled on a subsequent read?"
The highest timestamp wins.
"Do all zones work off the same system clock?"
Timestamps are provided by clients (i.e., your app server). They should be synchronized with e.g. ntpd (which is good practice anyway), but high precision is not required because if ordering matters you should be avoiding conflict either by using unique column names or by using external locking.
For example: if you have a list of users following you in a Twitter clone, you should give each follower its own column and there will be no way to lose data no matter how out of sync the clocks are.
If you have an admin tool for your website and two admins upload a new favicon "simultaneously," one update is going to win and it doesn't really matter which. Here, you do want your clocks synchronized but "within a few ms" is close enough.
If you are managing user registration and you want to allow creating account "jbellis" only if it doesn't already exist, you need a lock manager no matter how closely synchronzied your clocks are.
"Would stale data get returned?"
A node (a better unit to think about than a "zone") will not have data it missed during its downtime until it is sent that data by read repair, hinted handoff, or anti-entropy repair. In the meantime, it will reply to read requests with stale data; if you use a high enough consistencylevel read requests will wait for enough other replies to make sure you always see the most recent version anyway, which may mean not being able to fulfil requests if enough other replicas are down.
Otherwise, a low consistencylevel (e.g. ONE) implicitly means "I understand that the higher availability and lower latency I get with this lower consistencylevel means I'm okay with seeing stale data temporarily after downtime."
I'm not sure I understand all of the implications of the Cassandata consistency model with respect to data-agreement across multiple availability zones.
Given multiple zones, and given that the coordinator node in Cassandra has used a consistency level that does not require all zones to report back, but only a quorum, how would differences in zone data-state be reconciled on a subsequent read?
Do all zones work off the same system clock? Or does each zone have its own clock? If they don't work off the same clock, how are they synchronized so that timestamps can be compared during the "healing" process when differences are reconciled?
Let's say that a zone that does have accurate, up-to-date data is now offline, and a zone that was offline during a previous write (so it didn't get updated and contains stale data) is now back online. Would stale data get returned? Would the coordinator have any way to know the data were stale?
If you don't need to scale in the short term I'd go with Neo4j because it is designed to store networks like the one you described. (If you eventually do need to scale, maybe you can throw Gizzard in front of it or something. Good luck!)
Have you looked on Riak database? It has the same background as Cassandra, but you don't need to care about timestamp synchronization (they involve different method for resolving data status).
My first application was build on a Cassandra database. But I am now trying Riak because it is more suitable. It is not only the difference in keys (keys - values / super column - keys - values) but goes further with the document store feature.
It has a method to create complex queries using MapReduce. Cassandra does have this option using Hadoop, but it sounds difficult.
Further more it uses a well known and defined access protocol in http/s so it's easy to manage the server when you have a lot of traffic.
The only bad point is that is slower than Cassandra. But usually you will read records more than write (and Cassandra is optimised on writes, not reads) so the end result should be ok.