I intend to create a java program/service that continuously polls rss-feeds using the informa library 'poller' functionality. I want to be able to add,delete,update the rss-url's realtime, while the program is running. I have no prior experience with the informa library but I need it to potentially scale to a lot of rss-feeds.
Does anyone have have experience with the informa library for polling rss-feeds? What other method/libraries would you consider to poll a lot of rss-feeds (10.000+)?
What do you consider to be an accepted solution to control a running (console) java program. I was thinking about using a control port for sending commands. Are there other mechanisms more commonly used to achieve this functionality?
Please let me know if you need more specific information.
Kind regards,
Ivo
What do you consider to be an accepted solution to control a running
(console) java program. I was thinking about using a control port for
sending commands. Are there other mechanisms more commonly used to
achieve this functionality?
You can read the parameter from a .properties file. The only disadvantage with this is that the properties file will have to be read in each time you want to use that property, irrespective of whether the value has changed.
You can make use of JMX. This is a fairly nice concept in which you expose a bean to be manageable using the jconsole command (Java Management Extensions Console). Once done, you can then remotely inject values into a running JVM.
There is a nice example on Sun Oracle website that shows you how to do it.
Yes, a normal way to interact with a remote service would be a control port as you described it.
You can also control it via Database settings and create a thread which will poll for these DBs settings. The DB settings will be set via some web? UI.
If you plan on running one service with polling on one single machine I would rather recommend against it and set your service either on virtual machines or setup multiple instances of the service on one big machine with a big amount of memory. I have been using a com.sun.syndication library for feeds parsing/retrieving.
I don't want to be a captain obvious but I think it's easily achievable with a usual multi-threading application and Concurrent Queueing. If I got you correctly.
Thanks.
Related
I'm developing an application (Java & JavaFX) that writes/reads data (a file). The problem is I don't want to restrict user to run only one instance (of my app) at a time, as I really can't think of reliable way of doing that so it works on both Windows and Linuxes (e.g. server), heard of sockets and files - both are defective IMO. As user is able to run multiple instances, writing/reading data (from a file) seems really messy, because there's no guarantee that file locking will work reliably on Windows and Linuxes (FileLock documentation - click here).
To sum up: I can't restrict multiple instances of my app, but that leads to problem with writing/reading data (from a file).
Is there anything I missed? Maybe there's some other way to solve my problem I can't think of? How do the "big" popular programs handle that?
Suggested: Use a socket solution
You could follow the techniques outlined in an answer to:
JavaFX Single Instance Application
FAQ
Addressing some additional questions:
heard of sockets and files - both are defective IMO.
You state your opinion that using sockets to set up a single instance application won’t work well enough for you. You are in the best position to decide that.
For some apps which want to achieve a single instance, the socket-based or file-based solution outlined in the answer to the linked question or other comments will work well enough.
"What happens if more than one user try to run the application? Won't they conflict on opening the socket?"
Prevent launching multiple instances of a java application
And:
Also, I can't be sure if chosen port (fixed, since all instances should check for one port) is being used by some other applications/processes
You may be able to address some of these concerns by enhancing the socket-based solutions outlined in the linked questions.
Enhanced Socket Solution Outline
If you want, you can write an enhanced algorithm to deal with some of these issues.
When another app instance startup occurs, you try to connect to a current instance on a well-known socket.
Check the response to the connection.
If it doesn’t respond with the correct protocol response (e.g. matching user and app name) then increment the port by 2 and retry.
Test the response again until either:
You get a match for the app/user combo, then send a signal to that app to display itself.
OR
If you get no match, then create a new instance on the tested open port.
I'm not suggesting you do that, just explaining that it is possible.
Alternative: OS native service
There are also other OS-specific mechanisms for handling this such as Windows or Linux services which you can investigate if you want, those approaches are involved and vary by OS, so I won’t discuss them in detail here.
For the OS-specific solutions, you usually would:
Create a native package for your app (e.g. using jpackage)
Install it.
Have the installer config the app as a service
e.g. on linux, create an init.d script with a pid file configured via chkconfig.
The service launches on boot and stops on shutdown.
The app is then accessed via a tray icon or something similar
The means of interaction is often OS version specific.
Alternative: Allow multiple app instances but use a single database instance
You may also consider using a database rather than files for data storage, as a database system can help solve many of the concurrent access issues which can arise with file based solutions. Multiple clients can connect to the database, and the database and your app code can handle locks and collisions on the data access to ensure data integrity is contained. Using such a solution, there is no need to enforce that a single application instance is running for a user (at least from a data integrity perspective).
I'm looking for some simple methods to add some hooks into my Java backend code, like some counters or any other kind of value. These values should be easily accessible via an URL or API for monitoring or health-check. Also some tools to trigger an alert based on an unwanted condition that has arised in the server?
(Based on my and Gilbert's comments.)
First of all, there is no magic bullet solution. The actual work of gathering statistics, status values and the detection of "unwanted conditions" (or anomalous events) is down to the application code.
However, there some standard approaches to getting hold of this kind of information from a running application.
Instead of exposing the stats and status info via HTTP(s), you could use JMX to expose is, and use off-the-shelf JMX console softeare to access it.
The "unwanted conditions" / anomalous events requirement could be handled by using an off-the-shelf logging library. You then use an off-the-shelf external monitoring tool / system to scan for events and generate notifications.
Is it worth the effort? Depends on the nature of the server. (And on how much effort you can avoid by using monitoring infrastructure paid for or run by someone else.)
Firstly Cheers to all PROGRAMMERS [ Today = Programmers day :) ]
Secondly,
I'm working on a project where the specifications require using a server as a front end and an application in the back end. The project is an advanced smart home system. The server will handle commands coming from the client through the internet (let's say like a remote control from outside the house) and send them (through a channel of communication) to the application (planning on using JAVA application) which will handle the main logic like controlling hardware stuff (lights ...) , reading from a microphone (local mic) and accessing a database to act as a speech recognition system (offline).
Now I'm still in the planning phase and I'm not sure which technologies are the best for this project. I'm thinking to use Node.js or Apache as the server and a JAVA application as the back end and any SQL database for the application's SRS.
I hope this illustration demonstrates clearly how the system works:
The main question is:
What is the best way to make the Java application communicate with the server (communication channel [must be bidirectional]) ?
and Do you recommend a specific server other than the mentioned ones for this job ?
What crossed my mind so far:
1- JSP and servlets (making the server is the application too). But I don't want a server to handle the offline stuff and I'm not sure if java servlets can access hardware interface. I also want the server to be separate from making critical decisions (different layer for security reasons and since it won't be used as frequently as the local [offline] system).
2- Communication channel :
A- A shared file, but it's a bad idea since I don't want the application to check if the file contents changed (command received) or not from time to time (excessive operations).
B- A an inter-process-communication through a port (socket communication) seems the best solution but I don't know how that would turn in terms of operation cost and communication errors.
OS used : Linux Raspbian
EDIT:
I'm sure ZMQ+Apache is good enough for this task, but how is it in comparison to WebServices (like SOAP) ? Would WebServices be a better solution in terms of standard implementation and security ?
All related suggestions are welcomed, TQ
ZeroMQ is great for internal communications, or any other similar communication solutions.
For specifically your case, I can see that ZeroMQ would be a best fit.
Reasons:
You offline server have to be agnostic to web solution.
Communication can be reliable and bi-directional, possibly another patterns like (pub>sub, req<>res, etc).
Restarting any of sides would not require to restart the sockets (connection) on other side, as messages are queued.
Possibility to scale not just on same hardware, but as well to local area network or even through internet.
Big community of support. It might look a bit hard to get into, but in reality it is dead simple, just go to examples and once concept understood - it is very easy and neat to work with.
ZeroMQ has lots drivers for most popular languages, that includes Java and Node.js.
Considerations:
You need to think over packets and data will be sent. So some popular data protocols like XML or JSON is good way of thinking.
Responsibilities over different services - make sure they are not dependant on each other too much. Or if main offline server - is a core of system, make sure it does not depend on web facing service, so that web face can be removed/replaced/improved etc.
Few more points to think about:
Why Java, and what about modular approach? For example if you want to expand and scale - add more sensors into smart home solutions, then having one giant application would require to change it, it is harder to maintain as well as maintain different clients with own needs. Think modular way - some core functionality for offline stuff, but many aggregator processes that would talk to different sensors. This makes easier to support different setups and environments, as well maintain the system as a whole by improving independent components.
I'm writing a Java application that will run for a long time (essentially, it stays running between system restarts) and does a fair bit of intensive near real-time data processing. Data is delivered to the application, some work is done on that data, and it's then passed on for delivery.
I need to provide a way to inspect the inner workings of the application at runtime. I've already got a fairly well established but hand-carved telnet style interface that allows you to use a command line to ask questions about statistics, data queues, etc, but I'd like to move that to HTTP as I think it will allow me to provide a much richer picture of what's going on (showing load graphs, etc).
Are there any established Java frameworks that allow me to embed a web server and handle requests in a reasonable way? I don't really want to have to hand-carve a whole lot of HTML handling and response generation as it's just annoying background noise to the essence of what I need.
Are you sure you want to embed an HTTP interface?
A pretty common way of doing things like this is to set the Java application up with a restricted RMI interface. Then, you write a J2EE (JSP/Servlet) web interface which makes RMI calls to the application for data.
This setup avoids you having to implement the HTTP protocol (or any other interface protocol - RMI calls are just Java methods), and also separates data (the embedded J2SE program) from presentation (the user-interface J2EE program).
You may want to consider using jconsole and JMX (http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html).
They provide a graphical way to see what is going on inside your application and you can also setup controls to perform given actions through jconsole.
I am implementing a website using PHP for the front end and a Java service as the back end. The two parts are as follows:
PHP front end listens to http requests and interacts with the database.
The Java back end run continuously and responds to calls from the front end.
More specifically, the back end is a daemon that connects and maintain the link to several IM services (AOL, MSN, Yahoo, Jabber...).
Both of the layers will be deployed on the same system (a CentOS box, I suppose) and introducing a middle layer (for instance: using XML-RPC) will reduce the performance (the resource is also rather limited).
Question: Is there a way to link the two layers directly? (no more web services in between)
Since this is communication between two separate running processes, a "direct" call (as in JNI) is not possible. The easiest ways to do such interprocess communcation are probably named pipes and network sockets. In both cases, you'll have to define a communication protocol and implement it on both sides. Using a standard protocol such as XML-RPC makes this easier, but is not strictly necessary.
There are generally four patterns for application integration:
via Filesystem, ie. one producers writes data to a directory monitored by the consumer
via Database, ie. two applications share a schema or table and use it to swap data
via RMI/RPC/web service/any blocking, sync call from one app to another. For PHP to Java you can pick from the various integration libraries listed above, or use some web services standards like SOAP.
via messaging/any non-blocking, async operation where one app sends a message to another app.
Each of these patterns has pros and cons, but a good rule of thumb is to pick the one with the loosest coupling that you can get away with. For example, if you selected #4 your Java app could crash without also taking down your PHP app.
I'd suggest before looking at specific libraries or technologies listed in the answers here that you pick the right pattern for you, then investigate your specific options.
I have tried PHP-Java bridge(php-java-bridge.sourceforge.net/pjb/) and it works quite well. Basically, we need to run a jar file (JavaBridge.jar) which listens on port(there are several options available like Local socket, 8080 port and so on). Your java class files must be availabe to the JavaBridge in the classpath. You need to include a file Java.inc in your php and you can access the Java classes.
Sure, there are lots of ways, but you said about the limited resource...
IMHO define your own lightweight RPC-like protocol and use sockets on TCP/IP to communicate. Actually in this case there's no need to use full advantages of RPC etc... You need only to define API for this particular case and implement it on both sides. In this case you can serialize your packets to quite small. You can even assign a kind of GUIDs to your remote methods and use them to save the traffic and speed-up your intercommunication.
The advantage of sockets usage is that your solution will be pretty scalable.
You could try the PHP/Java integration.
Also, if the communication is one-way (something like "sendmail for IM"), you could write out the PHP requests to a file and monitor that in your Java app.
I was also faced with this problem recently. The Resin solution above is actually a complete re-write of PHP in Java along the lines of JRuby, Jython and Rhino. It is called Quercus. But I'm guessing for you as it was for me, tossing out your Apache/PHP setup isn't really an option.
And there are more problems with Quercus besides: the free version is GPL, which is tricky if you're developing commercial software (though not as tricky as Resin would like you to believe (but IANAL)) and on top of that the free version doesn't support compiling to byte code, so its basically an interpreter written in Java.
What I decided on in the end was to just exchange simple messages over HTTP. I used PHP's json_encode()/json_decode() and Java's json-lib to encode the messages in JSON (simple, text-based, good match for data model).
Another interesting and light-weight option would be to have Java generate PHP code and then use PHP include() directive to fetch that over HTTP and execute it. I haven't tried this though.
If its the actual HTTP calls you're concerned about (for performance), neither of these solutions will help there. All I can say is that I haven't had problems with the PHP and Java on the same LAN. My feeling is that it won't be a problem for the vast majority of applications as long as you keep your RPC calls fairly course-grained (which you really should do anyway).
Sorry, this is a bit of a quick answer but: i heard the Resin app server has support for integrating java and PHP.
They claim they can smash php and java together: http://www.caucho.com/resin-3.0/quercus/
I've used resin for serving J2ee applications, but not for its PHP support.
I'd be interested to hear of such adventures.
Why not use web service?
Make a Java layer and put a ws access(Axis, SpringWS, etc...) and the Php access the Java layer using one ws client.
I think it's simple and useful.
I've come across this page which introduces a means to link the two layers. However, it still requires a middle layer (TCP/IP). Moreover, other services may exploit the Java service as well because it accepts all incoming connections.
http://www.devx.com/Java/Article/20509
[Researching...]