Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
My use case is to develop a telecom billing in Java. Knowing that i must calculate 60 000 bills per day, i need to distribute the load of the calculation on several(three or four) servers.
That is called Clustering ( correct me if it's wrong).
My Solution is to develop a middleware that will distribute for each server a list of client who should be charged, and then each server will calculate the bill and generate a PDF file.
Could you give me some more ideas, for example which Java class do i need, or which methods shall i use.
Thank you
if you use jms (maybe as part of using j2ee?) you could simply use a jms queue for this - have several consumers (mdbs in j2ee) on every node, and send the list of clients to process to the queue.
the queue will guarantee that every message (==client) will be handled by one and only one listener, and since each node will have a limited number of listeners you get work distribution this way.
try apache hadoop.
http://hadoop.apache.org/
It will be perfect for this kind of divide and conquer tasks.
Spring batch will be a good fit for that: http://projects.spring.io/spring-batch/
It has support for automatic restart of jobs and partitioning ( distribution over many machines ).
There is an entire chaptr in the docs dedicated to scaling/distribution: http://docs.spring.io/spring-batch/2.2.x/reference/html/scalability.html
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I am building an centralized inventory system using java(swing and servlets). I have local data stored in desktop application which needs to be in sync with an server.
Requirement is:-
1) When local stores are online ,data must be synchronized with server in real time.
2) Local store will have access to their data even if they go offline, but when they go online all local changes must be reflected on server.
I got across microsoft sync framework which is exactly what I need. Is there something equivalent in java?
There are a number of Open Source Replication and Synchronization Tools Written in Java:
Of those listed the following might handle your "offline" requirement:
SymmetricDS – SymmetricDS is a web based, database independent, data synchronization system. Beyond replicating tables between relational databases the software incorporates functionality to handle a large number of databases, manage low-bandwidth connections and work within periods of network outage. The system supports two way replication, guaranteed delivery and multiple active schemas.
I'm sure the others are worth a quick look as well.
See also Strategy for Offline/Online data synchronization for some great discussion on things to think about/watch out for.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
Is there a way to find an online Hadoop database and practice on it using Java?
I found that you can practice on www.gethue.com, but I don't think you can do it using java.
Thank you
You can try Cloudera Live.
It's in beta, but seems to work pretty well.
I made a small list of free offers enabling you to manage your own hadoop cluster. It's not technically an available database, but you can fill these clusters with the data you want.
Here is the list :
Microsoft Azure HDInsight : they offer you 150€ to spend on their products. You can rent a Hadoop cluster and work on it.
Qubole : they give you preconfigured Hadoop clusters, you have 75 computing hours for free
Joyent : you can have one VM for free for a year.
You may also try amazon's Elastic Map Reduce, although I'm not sure this specific offer is included in their free trial. An advantage of using it is you can access free datasets more easily (for instance, this one).
Please also note that all these services (except Qubole) require a credit card for registration.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I have a Java application. I can expose it using web services or REST (JAX-WS or JAX-RS).
Now I actually want to run it "in a cloud" and expose it as a service. I have read around that there are services e.g. Apigee, Rackspace, Google App Engine... The idea is that I don't want to worry about scale and performance. I want that handled by the host.
What are the options for Java?
Thanks,
David.
After a bit of homework, here's what I am gathering:
This is really about Java PaaS offerings (platform as a service).
In addition to what I previously mentioned,
Google App Engine
Rackspace
Apigee
it's worth adding:
Jelastic
CloudBees
There's also a guide - albeit old - from InfoQ that can be read here.
Depends on amount of money you can spend.
This cloud hosting seems interesting.
There's Heroku too, or even AWS.
Amazon AWS provides auto-scaling features that you can configure so you don't have to "worry about scale" day to day, though you do have to set it up in the first place (you will also have to monitor your bill in case you are scaling big-time ;) ). It works well and provides decent monitoring/visualization if you are happy to do the set up.
I can't say whatthe other systems you are investigating have to offer in terms of automatic scaling though.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm currently using NewRelic (http://newrelic.com/) for monitoring my java applications. Point is I would like to monitor Zookeeper, Hornetq and other java-based application that run without server or application container.
Looks like NewRelic does not support this things..
Could you please tell me if there is a good universal monitor tool that I can use for this purpose ? (rather than JMX / Jconsole...)
You might be interested in Ganglia: http://ganglia.sourceforge.net/
For Zookeeper: https://github.com/andreisavu/zookeeper-monitoring/tree/master/ganglia
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
what is a better nosql database for creating a system to record advertisement data for about 50 to 200 millions insert per day, the aggregation of the data will be used to show the pattern of how users engage with the ads. I really like MongoDB but it seems that major industry players are picking Riak for the job. It seems that Mongo had to flush some caveats in last 2 releases and the current version seems to be pretty good for the job, any idea?
It seems mongodb with hadoop (http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-hadoop/ ) fits your data requirements. You can store data in mongodb and run aggregation jobs (map/reduce) on hadoop cluster.
I'd use Java Berkeley DB from Oracle.
Very powerful and easy to use Open source but not free.
This is not a type of question that could be answered as Product1 or Product2. It is just too small amount of information given. There is no info about the environment, where the system will run, what type of information will be inserted, how are you going to aggregate it.
The best way is to try:
write a test using Product1,
write a the same thing with a Product2
start inserting the data which looks as close as possible to the
data you are assuming to get in real environment
make measurements of speed and whatever factors you need
and only based on that you will be able to determine what suits you