What is the difference between a Thread and a Process in the Java context?
How is inter-Process communication and inter-Thread communication achieved in Java?
Please point me at some real life examples.
The fundamental difference is that threads live in the same address spaces, but processes live in the different address spaces. This means that inter-thread communication is about passing references to objects, and changing shared objects, but processes is about passing serialized copies of objects.
In practice, Java interthread communication can be implemented as plain Java method calls on shared object with appropriate synchronization thrown in. Alternatively, you can use the new concurrency classes to hide some of the nitty-gritty (and error prone) synchronization issues.
By contrast, Java interprocess communication is based at the lowest level on turning state, requests, etc into sequences of bytes that can be sent as messages or as a stream to another Java process. You can do this work yourself, or you can use a variety of "middleware" technologies of various levels of complexity to abstract away the implementation details. Technologies that may be used include, Java object serialization, XML, JSON, RMI, CORBA, SOAP / "web services", message queing, and so on.
At a practical level, interthread communication is many orders of magnitude faster than interprocess communication, and allows you to do many things a lot more simply. But the downside is that everything has to live in the same JVM, so there are potential scalability issues, security issues, robustness issues and so on.
A thread can access memory inside a process, even memory that could be manipulated by another thread within the same process. Since all threads are internal to the same running process, they can communicate more quickly (because they don't need the operating system to referee).
A process cannot access memory inside another process, although you can communicate between processes through various means like:
Network packages.
Files
Pipes
Shared Memory
Semaphores
Corba messages
RPC calls
The important thing to remember with process to process communication is that the communication must be managed through the operating system, and like all things which require a middle man, that adds overhead.
On the downside, if a thread misbehaves, it does so within the running process, and odds are high it will be able to take down all the well behaving threads. If a process misbehaves, it can't directly write into the memory of the other processes, and odds are that only the misbehaving process will die.
Inter-Thread Communication = threads inside the same JVM talking to each other
Inter-Process Communication (IPC) = threads inside the same machine but running in different JVMs talking to each other
Threads inside the same JVM can use pipelining through lock-free queues to talk to each other with nanosecond latency.
Threads in different JVMs can use off-heap shared memory (usually acquired through the same memory-mapped file) to talk to each other with nanosecond latency.
Threads in different machines can use the network to talk to each other with microsecond latency.
For a complete explanation about lock-free queues and IPC you can check CoralQueue.
Disclaimer: I am one of the developers of CoralQueue.
I like to think of a single instance of a JVM as a process. So, interprocess communication would be between instances of JVM's, for example, through sockets (message passing).
Threads in java implement Runnable and are contained within a JVM. They share data simply by passing references in the JVM around. Whenever threads share data you almost always need to protect the data so multiple threads don't clobber each other. There are many mechanisms for protection that all involve preventing multiple threads from entering critical sections of code.
Related
When implementing a server, we can delegate one client request to one thread. I read that problem with this approach is that each thread will have its own stack and this would be very "expensive". Alternative approach is that have server be single threaded and implement all client requests on this one server thread with I/O requests as non-blocking request. My doubt is that if one server thread is running multiple client requests simultaneously, won't server code have instruction pointer, set of local variables, function calls stacks for each client request, then won't this again be "expensive" as before. How are we really saving?.
I read that problem with this approach is that each thread will have its own stack and this would be very "expensive".
Depends on how tight your system resources are. The typical JVM stack-space allocated per thread defaults to 1mB on many current architectures although this can be tuned with the -Xss command line argument. How much system memory your JVM has at its disposal and how many threads you need determines if you want to pay the high price of writing the server single threaded.
My doubt is that if one server thread is running multiple client requests simultaneously, won't server code have instruction pointer, set of local variables, function calls stacks for each client request, then won't this again be "expensive" as before
It will certainly will need to store per request context information in the heap but I suspect that it would be a lot less than 1mB worth of information to hold the variables necessary to service the incoming connections.
Like most things, what we are really competing against when we look to optimize a program, whether to reduce memory or other system resource use, is code complexity. It is harder to get right and harder to maintain.
Although threaded programs can be highly complex, isolating a request handler in a single thread can make the code extremely simple unless it needs to coordinate with other requests somehow. Writing a high performance single threaded server would be much more complex than the threaded version in most cases. Of course, there would also be limits on the performance given that you can't make use of multiple processors.
Using non blocking I/O, A single I/O thread can handle many connections. The I/O thread will get notification when:
client wants to connect
the write buffer of the socket of the connection has space when the write buffer of the socket was full the previous round.
the read buffer of the socket of the connection has data available for reading
So the thread makes use of event-multiplexing to serve the connections concurrently using a selector. A thread waits for a set of selection-keys from the selector, and the selection key contains the state of the event you have registered for and you can attach user data like a 'session' to the selection-key.
A very typical design pattern used here is the reactor pattern.
But often you want to prevent blocking the I/O thread with longer running requests. So you offload the work to a different pool of threads. Then the reactor changes to the proactor pattern.
And often you want to scale the number of I/O threads. So you can have a bunch of I/O threads in parallel.
But the total number of threads in your application should remain limited.
It all depends on what you want. Above are techniques I frequently used while working for Hazelcast.
I would not start to write all this logic from scratch. If you want to make use of networking, I would have a look at Netty. It takes care of most of the heavy lifting and has all kinds of optimizations built in.
I'm not 100% sure if the a thread that doesn't write to its stack will actually consume 1MB of physical memory. In Linux the (shared) zero-page is used for a memory allocation, so no actual page frame (physical memory) is allocated unless the stack of the thread is actually written to; this will trigger a copy on write to do the actual allocation of a page-frame. Apart from saving memory, this also prevents wasting memory bandwidth on zeroing out the the stack. Memory consumption of a thread is one thing; but context switching is another problem. If you have many more threads than cores, context switching can become a real performance problem.
Can we run multiple processes in one JVM? And each process should have its own memory quota?
My aim is to start new process when a new http request comes in and assign a separate memory to the process so that each user request has its own memory quota - and doesn't bother other user requests if one's memory quota gets full.
How can I achieve this?
Not sure if this is hypothetical.
Short answer: not really.
The Java platform offers you two options:
Threads. And that is the typical answer in many cases: each new incoming request is dealt with by a separate thread (which is probably coming out of a pool to limit the overall number of thread instances that get created/used in parallel). But of course: threads exist in the same process; there is no such thing as controlling the memory consumption "associated" by what a thread is doing.
Child processes. You can create a real process and use that to run whatever you intend to run. But of course: then you have an external real process to deal with.
So, in essence, the real answer is: no, you can't apply this idea to Java. The "more" Java solution would be to look into concepts such as application servers, for example Tomcat or WebSphere.
Or, if you insist on doing things manually; you could build your own "load balancer"; where you have one client-facing JVM; which simply "forwards" requests to one of many other JVMs; and those "other" JVMs would work independently; each running in its own process; which of course you could then "micro manage" regarding CPU/memory/... usage.
The closest concept is Application Isolation API (JSR-121) that AFAIK has not been implemented: See https://en.wikipedia.org/wiki/Application_Isolation_API.
"The Application Isolation API (JSR 121) provides a specification for isolating and controlling Java application life cycles within a single Java Virtual Machine (JVM) or between multiple JVMs. An isolated computation is described as an Isolate that can communicate and exchange resource handles (e.g. open files) with other Isolates through a messaging facility."
See also https://www.flux.utah.edu/janos/jsr121-internal-review/java/lang/isolate/package-summary.html:
"Informally, isolates are a construct midway between threads and JVMs. Like threads, they can be used to initiate concurrent execution. Like JVMs, they cause execution of a "main" method of a given class to proceed within its own system-level context, independently of any other Java programs that may be running. Thus, isolates differ from threads by guaranteeing lack of interference due to sharing statics or per-application run-time objects (such as the AWT thread and shutdown hooks), and they differ from JVMs by providing an API to create, start, terminate, monitor, and communicate with these independent activities."
I have basic idea about concurrency but I have a confusion about the following architecture. I think it is concurrent but my colleague thinks it is not. The architecture is as follows:
I have multiple robots which publish its data to its individual gateways and there's another java service which listens on the gateways. The service creates a new thread to listen to each gateway.
My understanding is that the service is performing concurrent execution but my colleague says this is not concurrent as concurrency involves sharing of hardware.
Appreciate if some one can clarify or elaborate on this topic.
My understanding is that the service is performing concurrent execution but my colleague says this is not concurrent as concurrency involves sharing of hardware.
TL/DR: Words are squishy. That's why we have code.
"Concurrent" simply means two or more things happening at the same time. As it applies to computation, true concurrency means two or more threads of execution running at the same time, which requires separate hardware. That certainly can be separate cores of the same CPU or separate CPUs in the same chassis, so that there is some degree of shared hardware. It can also be separate cores in different chassis, however, such as in a computational cluster, though perhaps this is where your colleague is drawing his line. Such a line would be pretty arbitrary, though.
In contrast, long before it was common for even servers to feature multiple CPU (core)s, many computer systems implemented one flavor or another of multitasking, whereby multiple tasks can all be in progress at the same time by virtue of the operating system allotting slices of CPU time to each and switching them in and out. All modern general-purpose operating systems still do this. On a single core, however, this provides only simulated concurrency, because at any given instant in time, only one computation is actually making progress.
Your colleague does have a point, however, that multiple, spatially distributed robots all operating at the same time without coordination is a bit beyond what people usually mean when they talk about concurrent computation. Certainly such robots are operating concurrently, in the general-use sense of "at the same time", but it's a bit of a stretch to characterize them as participating in a concurrent computation.
The server that allocates a separate thread to handle communication with each robot may thereby be performing a concurrent computation. But as long as we're splitting hairs, do recognize that communication over a single network interface is serialized, so unless your server has multiple network interfaces, the actual communication cannot be truly concurrent. If the server is primarily just recording the data as it arrives, as opposed to incorporating it into an ongoing concurrent computation, then it would be potentially misleading to describe it as performing a concurrent operation.
Even by your colleague's definition, this is a concurrent system since there are multiple threads executing on the hardware on which the service resides.
For what reasons would one choose several processes over several threads to implement an application in Java?
I'm refactoring an older java application which is currently divided into several smaller applications (processes) running on the same multi-core machine, communicating which each other via sockets.
I personally think this should be done using threads rather than processes, but what arguments would defend the original design?
I (and others, see attributions below) can think of a couple of reasons:
Historical Reasons
The design is from the days when only green threads were available and the original author/designer figured they wouldn't work for him.
Robustness and Fault Tolerance
You use components which are not thread safe, so you cannot parallelize withough resorting to multiple processes.
Some components are buggy and you don't want them to be able to affect more than one process. Say, if a component has a memory or resource leak which eventually could force a process restart, then only the process using the component is affected.
Correct multithreading is still hard to do. Depending on your design harder than multiprocessing. The later, however, is arguably also not too easy.
You can have a model where you have a watchdog process that can actively monitor (and eventually restart) crashed worker processes. This may also include suspend/resume of processes, which is not safe with threads (thanks to #Jayan for pointing out).
OS Resource Limits & Governance
If the process, using a single thread, is already using all of the available address space (e.g. for 32bit apps on Windows 2GB), you might need to distribute work amongst processes.
Limiting the use of resources (CPU, memory, etc.) is typically only possible on a per process basis (for example on Windows you could create "job" objects, which require a separate process).
Security Considerations
You can run different processes using different accounts (i.e. "users"), thus providing better isolation between them.
Compatibility Issues
Support multiple/different Java versions: Using differnt processes you can use different Java versions for your application parts (if required by 3rd party libraries).
Location Transparency
You could (potentially) distribute your application over multiple physical machines, thus further increasing scalability and/or robustness of the application (see #Qwe's answer for more Details / the original idea).
If you decide to go with threads you will restrict your app to be run on a single machine. This solution doesn't scale (or scales to some extent) - there are always hardware limits.
And different processes communicating via sockets can be distributed between machines, so that you could add virtually unlimited number or them. This scales better at the cost of slow communication between processes.
Deciding which approach is more suitable is itself a very interesting task. And once you make the decision there's no guarantee that it will look stupid to your successors in a couple of years when requirements change or new hardware becomes available.
I have a C program that will be storing and retrieving alot of data in a Java store. I am putting alot of stress in my C program and multiple threads are adding and retrieving data from Java store. How will java handle such load? Because if there is only one main thread running JVM and handling all the requests from C, then it may become bottleneck for me. Will Java create multiple threads to handle the load, or is it programmers job to create and later on abort the threads?
My java store is just a Hashtable that stores the data from C, as is, against a key provided.
You definitely want to check the jni documentation about threading, which has information around attaching multiple native threads to the JVM. Also you should consider which Map implementation that you need to use. If accessing from multiple Hashtable will work, but may introduce a bottle neck as it is synchronized on every call, which will effectively mean a single thread reading or writing at a time. Consider the ConcurrentHashMap, which uses lock striping and providers better concurrent throughput.
A couple of things to consider if you are concerned about bottlenecks and latency.
On a heavily loaded system, locking can introduce a high overhead. If the size of you map and the frequency of write allows, consider using an immutable map and perform a copy on write approach, where a single thread will handle writes by making updates to a copy of the map and replacing the original with the new version (make sure the reference is a volatile variable). This will allow reads to occur without blocking.
Calling from C to Java via JNI will probably become a bottle neck too, its not as fast as calling in the other direction (Java to C). You can pass Direct ByteBuffers through to Java that contain references to the C data structures and allow Java to call back down to C via the Direct ByteBuffer.
Plain Java requires that you write your own threading.
If you are communicating to java via web services it's likely that the web container will manage threads for you.
I guess you are using JNI, so then the situation is potentially more complex. Depending upon exactly how you are doing your JNI calls you can get at multiple threads in the JVM.
I've got to ask ... JNI is pretty gnarly and error prone, all too easy to bring down the whole process and get all manner of mysterious errors. Are there not C libraries containing a HashTable you could use? Or even write one, it's got to be less work than doing JNI.
I think this depends on the java code's implementation. If it proves not to thread, here's a potentially cleaner alternative to messy JNI:
Create a Java daemon process that communicates with your store, which INTERNALLY is threaded on requests, to guarantee efficient load handling. Use a single ExecutorService created by java.util.concurrent.Executors to service a work queue of store/retrieve operations. Each store/retrieve method call submits a Callable to the work queue and waits for it to be run. The ExecutorService will automagically queue and multithread the store/retrieval operations. This whole thing should be less than 100 lines of code, aside from communications with the C program.
You can communicate with this Java daemon from C using inter-process communication techniques (probably a socket), which would avoid JNI and let one Java daemon thread service numerous instances of the C program.
Alternately, you could use JNI to call the basic store/retrieve operations on your daemon. Same as currently, except the java daemon can decorate methods to provide caching, synchronization, and all sorts of fancy goodies associated with threading.