Develop Reference

Inter-process-communication between a Java application and a local server

Firstly Cheers to all PROGRAMMERS [ Today = Programmers day :) ]
Secondly,
I'm working on a project where the specifications require using a server as a front end and an application in the back end. The project is an advanced smart home system. The server will handle commands coming from the client through the internet (let's say like a remote control from outside the house) and send them (through a channel of communication) to the application (planning on using JAVA application) which will handle the main logic like controlling hardware stuff (lights ...) , reading from a microphone (local mic) and accessing a database to act as a speech recognition system (offline).
Now I'm still in the planning phase and I'm not sure which technologies are the best for this project. I'm thinking to use Node.js or Apache as the server and a JAVA application as the back end and any SQL database for the application's SRS.
I hope this illustration demonstrates clearly how the system works:
The main question is:
What is the best way to make the Java application communicate with the server (communication channel [must be bidirectional]) ?
and Do you recommend a specific server other than the mentioned ones for this job ?
What crossed my mind so far:
1- JSP and servlets (making the server is the application too). But I don't want a server to handle the offline stuff and I'm not sure if java servlets can access hardware interface. I also want the server to be separate from making critical decisions (different layer for security reasons and since it won't be used as frequently as the local [offline] system).
2- Communication channel :
A- A shared file, but it's a bad idea since I don't want the application to check if the file contents changed (command received) or not from time to time (excessive operations).
B- A an inter-process-communication through a port (socket communication) seems the best solution but I don't know how that would turn in terms of operation cost and communication errors.
OS used : Linux Raspbian
EDIT:
I'm sure ZMQ+Apache is good enough for this task, but how is it in comparison to WebServices (like SOAP) ? Would WebServices be a better solution in terms of standard implementation and security ?
All related suggestions are welcomed, TQ

ZeroMQ is great for internal communications, or any other similar communication solutions.
For specifically your case, I can see that ZeroMQ would be a best fit.
Reasons:
You offline server have to be agnostic to web solution.
Communication can be reliable and bi-directional, possibly another patterns like (pub>sub, req<>res, etc).
Restarting any of sides would not require to restart the sockets (connection) on other side, as messages are queued.
Possibility to scale not just on same hardware, but as well to local area network or even through internet.
Big community of support. It might look a bit hard to get into, but in reality it is dead simple, just go to examples and once concept understood - it is very easy and neat to work with.
ZeroMQ has lots drivers for most popular languages, that includes Java and Node.js.
Considerations:
You need to think over packets and data will be sent. So some popular data protocols like XML or JSON is good way of thinking.
Responsibilities over different services - make sure they are not dependant on each other too much. Or if main offline server - is a core of system, make sure it does not depend on web facing service, so that web face can be removed/replaced/improved etc.
Few more points to think about:
Why Java, and what about modular approach? For example if you want to expand and scale - add more sensors into smart home solutions, then having one giant application would require to change it, it is harder to maintain as well as maintain different clients with own needs. Think modular way - some core functionality for offline stuff, but many aggregator processes that would talk to different sensors. This makes easier to support different setups and environments, as well maintain the system as a whole by improving independent components.

How to achieve some measure of "privilege separation" with Java web-server?

I'm trying to be proactive around security on my Jetty web-server boxes -- especial with regards to storing SSL key information although I'd like a generic solution. Apache uses privilege separation so that it starts as root so it can read the protected SSL key files (and other secure configuration) and then switches to some common user to actually server HTTP requests. But Java has no mechanism for doing this.
Any recommendations around how to achieve the same level of security in a Java web application? My requires include:
Secret information should only be readable by root.
Any passwords which unlock keys and the like should not be configured into the code so that someone with the same user level permissions as the server can't get them easily.
I'm running under Amazon EC2 so I want the security to be as automatic as possible -- i.e. no interactive password entering by operators.
One possibility would be to use ~LDAP to separate the secret information from the application and only bake the LDAP access username/password into the application. But I'm looking for a better solution.
Thanks for any information.
Edit:
I'd hoped for solutions that covered SSL but took into account other secrets that I wanted to limit access to. I did not make that clear enough in my initial post.

The apache technique you described is provided by the optional jetty-setuid features.
See http://www.eclipse.org/jetty/documentation/current/setuid.html

As soon as you bake anything like a password into source (which is stored on disk), you've circumvented security. So, storing the information in LDAP isn't going to help.
I'm not convinced the setuid feature is going to help either, in that it is there purely for accessing ports in the networking code, and might not do the setuid at the correct time (after opening the SSL files). Of course, you could test that by protecting the files as root and see if it can open them...if so, you're golden and Joakim's answer is the best option.
What we do is set up a simple apache or nginx server to front the JVM through a proxy, then run jetty under it's own UID. Then you can take advantage of the setuid SSL security that is already well-tested in either of those servers. We also have some other requirements that this also helps solve, but I would probably choose to do it this way even if we didn't.
The nginx config is also pretty darn simple:
server {
listen 192.168.1.1:443;
server_name www.mydomain.com;
index index.html index.htm;
root /usr/share/nginx/html;
ssl on;
ssl_certificate /etc/nginx/conf.d/ssl/server.crt;
ssl_certificate_key /etc/nginx/conf.d/ssl/server.key;
access_log /var/log/nginx/ssl.access.log main;
ssl_session_timeout 5m;
ssl_protocols SSLv2 SSLv3 TLSv1;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location /AppPath {
proxy_pass http://jettyhost:8080/AppPath;
}
}

I would suspect that, because of the multi-platform nature of Java, there hasn't been much emphasis on Unix-based techniques like what exists in Apache Httpd, since these may not necessarily be applicable to all platforms on which Java can run.
In Java, privilege separation is in-built, by the means of the security manager. Whether this is as good as the root/non-root separation, I'm not sure (there can always be bugs). However, its policies are, in principle, capable of expressing more subtle access rules than just the distinction between a root and non-root users.
Jetty had this, but this was apparently dropped in Jetty 9 (you can still use a security manager, but you may have to write your own policies and do a bit more of the work to implement them in the container).
The Jetty Policy document also states:
Normally a user trusts the application they are developing or are trusting enough to deploy a webapp into an instance of jetty. If you don't know that you need to use the security manager setup, you probably don't.
I'm not sure I'd agree with that. Indeed, if there's a webapp I suspect to be malicious, I won't run it anyway, but using a security manager is also about containing potential security bugs. Anyone can write bugs, even good programmers. Having a mechanism that restrict possible actions by webapps is certainly a good thing.
A sensible policy would certainly prevent webapps from accessing the config files and the keystores.
I would also argue that this webapp separation is also at the core of the notion of "container" (although security is only one of the purposes of this separation, it seems to have been lost).
In fairness, it's not as straightforward as the separation offered by the root/forking mechanism in Apache Httpd. The power of Java security policies also brings complexity. I suppose these features are generally not well understood, and thus little used. Using Apache Httpd (or Nginx, or others) as a reverse proxy tends to be a simpler solution for the purpose of protecting the private keys.
Another way you could look into is using a PKCS#11 keystore. This is supported by the JRE. Hardware Security Modules would help prevent your private keys to be copied. (From a Java point of view, you get a PrivateKey instance that delegates the cryptographic operations to the PKCS#11 library, but from which you cannot extract the private data at all.)
Of course, this is a problem when you don't have access to the hardware, but there are software implementations of this (looking up "Software HSM" might seem odd, but it will bring a few results). Not all of them will let you have the separation you're after, but some should (effectively, by communicating with another entity holding the private key, which you could potentially run as another user). I haven't tried it, but this one might be of interest.

Although I appreciate both #Joakim and #Tony's answers, I was hoping for a more generic solution that covered general password protection without JNI/Jetty specific features and more generic than just SSL secret key protection.
The best solution I can come up with is a small C wrapper program that was setuid root. It would:
Starts up and read in a collection of secret information from root protected files on disk into memory. It should immediately encrypt the secret information in memory (see below).
Switches from root to the unprivileged user running the application.
Forks and exec's JVM with the appropriate application arguments.
Writes the encryption key and the encrypted passwords in via STDIN.
When the JVM boots, it immediately reads in the encrypted secret information from STDIN.
One they are read the wrapper application will terminate.
As an extension, the small C wrapper could stay running and provide access to the JVM to system resources by using a simple line based protocol over STDIN/STDOUT. This would give the JVM access to other protected resources on the system in a controlled manner.
Here are some other thoughts on possible solutions.
A service started at boot time by init.d script that runs as root and serves a collection of secret keys to the JVMs starting up by some web service or though some FIFO file or something. After the first request it would shutdown or after some number of seconds after boot.
LDAP is certainly better than having the secret foo on the box itself -- readable by the application user. As an alteration to the solution above, the setuid program could inject the LDAP password into application so it would not live in user readable space.
As always, both applications would need to protect the passwords in memory. Storing them in system sockets or splitting them up into noncontiguous memory blocks is always a good idea. You could also generate a secret key and encrypt them in memory as well.

Looking for simple way of representing database contents as filesystem (Windows)

I have a Document Management System that stores documents in a database. I'm looking for a simple way (not too much and complicated protocol to implement) to show the database as a drive in windows (so it can be browsed and manipulated using any windows program, like explorer or office).
I have something in mind that I provide some kind of network share and that can be mounted as a drive in windows. Unfortunately all candidate network protocols for file sharing seem to require substantial effort to implement.
I first considered CIFS, but after reading up on that quickly decided that its BY FAR to complicated for me to implement. Next thougt was NFS, but its not supported by Windows (XP) natively and also seems quite complicated to implement.
FTP might be an option, but implementing an FTP server is again much more complicated than I naively expected.
There might be a simpler protocol to use I haven't thought of.
Is there anything I can (ab)use easily for this purpose?
Ideally I want some kind of (pure Java) premade server where I could easly strip out the part that accesses a local file system and replace it with my own code accessing the database OR a protocol simple enough that I can implement it myself reasonably quickly and more importantly, compatible and reliable.

First, you need to make the correct bindings between your DBS and the database, and define/write an API in your program that will describe an easy access to the needed resources. Writing an API Will allow to maximize inter-operablity of your solution with other services or plugins you might want to add later.
After that, you should serve this services to clients through a permissive and robust protocol such as WebDav which stands for Web-based Distributed Authoring and Versioning. He is natively supported by Windows, you can this way interact with every services implementing WebDav (Windows, most web browsers ...), and you can of course also mount this kind of services as a virtual drive. Plus, it is also supported on Linux and MacOS X, I believe nativ'ely but i'm not sure. In fact, WebDav is an extension to HTTP, and is described in the RFC 4918.
Basically every HTTP (which handles server side response management) library for Java could be used to implement WebDav, if you havé some time and want to do it yourself.
To implement WebDav in a way and with an acceptable effort, I searched for some Java librairies on the web and found these ones, it's now up to you to decide which one really fits your needs :
Milton http://milton.ettrema.com/index.html
A list of some of the existing implementations of the WebDav protocol into open source projects on WebDAV.org (you can find there some pretty amazing projects such as The Jakarta Slide project although I think it is not supported anymore and other projects/librairies that shows the importance of WebDav today). http://www.webdav.org/projects/

Java networking?

First off, before I ask, i would like to point out that this question is for education. I want to know to expand my understanding of Java and network security (what little there is).
How could you use Java for network security and counter attacks? I have been using server/sockets for a while now (for non system security stuffs), but I don't quite understand what I'm doing. Naturally, I should learn up on networking, but where to start? There is a protocol for everything, heck there are protocols to have protocols. To further expand, how could you use Java to say, port sniff, catch packets or kill/open a port remotely?
I guess to phrase the question more adequately; does anyone know of any good sources that I could look at to get a more in depth look/study of how Java handles network security and counter hacking and malware containment?

I think the best thing to do would be to learn concepts, then worry about using Java to implement the concepts later on. There are some gaps in your understanding (for example, I don't even know what "open a port remotely" might mean) and the best thing to do would be to solidify your understanding of how networks work first.
I don't really have a list of network security texts I can recommend -- probably someone else will! -- but IMHO it might not hurt to start with a classic like Steven's "UNIX Network Programming" to shore up the fundamentals, if you can find a copy.

how could you use java to say, port
sniff, catch packets or kill/open a
port remotely?
You can't use Java to sniff ports.
You can't use Java to catch packets.
You can't use Java to kill/open a port remotely.
how Java handles network security and
counter hacking and malware
containment?
Java doesn't handle network security other than internally for its own applications via the security sandbox.
Java doesn't handle counter hacking.
Java doesn't handle malware containment other than internally for its own applications via the security sandbox and bytecode verifier.
One of those things above can be done via an add-on to Java, but basically Java isn't the correct tool for this job.

Interprocess Communication between C++ app and Java App in Windows OS environment

We have a C++ application on Windows that starts a java process. These two apps need to communicate with each other (via snippets of xml).
What interprocess communication method would you choose, and why?
Methods on the table for us are: a shared file(s), pipes and sockets (although I think this has some security concerns). I'm open to other methods.

I'm not sure why you think socket-based communication would have security concerns (use SSL). It is often a very good approach as it is language agnostic, assuming that you have a well-defined communication protocol. Have a look at Google's protocol buffers, for example - they generate the required Java classes and streams.
In my experience, file systems (especially network file systems) are not well suited to such communication as they are not necessarily tuned for messaging (I've seen caching issues result in files being not picked up by the target process for example).
Another option is a messaging layer (AMQ or Tibco for example) although this will likely involve a greater administrative overhead (plus expertise) to set up.
Personally I would opt for a pure-socket approach because of its flexibility and simplicity. You will be in complete control.

I've used named pipes for communication between C# and a cross-platform c++ app and had nothing but good results. Barring that sockets is definitely the way to go.

Sockets are nice. They give you the ability to very easily create a blackbox testing layer around each component, as well as run each component on its own machine.
Security is definitely a concern, but there are a good range of options depending on how important it is. You can use SSL, custom handshaking, password protected logins and firewalls to help secure it.
Edit:
Not something I'd recommend, but there's also shared memory using JNI. Just thought I'd mention it because it's not on your list.

Ice is pretty cool :)