What is up with nio channels ? There were some nice talks when it was added to java but I still don't see people using it in their applications.
Is there something wrong with it, or am I just not encountering people who use it?
Any nice examples as to why I should bother using it at all ?
Thanks
You're asking about channels, but channels only make sense within the general framework of using the (relatively) new nio capabilities as a whole.
My guess is that of the many, many Java applications out in the world, not many need the capabilities of nio. The usual "business" process read streams and/or files... nothing special.
That said, the Apache folks have recently rewritten their core Java libraries ( http://hc.apache.org/ ) to use nio, and claim some impressive performance benefits in some cases.
nio also lets you do stuff like memory-mapping files, and this can allow an application to do very fast random access to the file. Again, only some special applications need this, and that's probably why you don't see a lot of it used.
Apache Mina is a great networking library and uses NIO.
Apache MINA is a network application framework which helps users develop high performance and high scalability network applications easily. It provides an abstract · event-driven · asynchronous API over various transports such as TCP/IP and UDP/IP via Java NIO.
Net4J, a signaling platform/framework, makes heavy use of NIO channels. (One part of Net4J basically provides a convenience API to NIO channels.)
Related
As we had known, If we want to use traditional IO to construct server, it must block somewhere, so we had to use loop or one thread one socket mode, So nio seem it is better choice. So I want know if the nio is better choice forever?
IMHO, Blocking IO is generally the simplest to use, and unless you have a specific requirement which demands more from your system, you should stick with simplest option.
The next simplest option is blocking NIO, which I often prefer if I want something more efficiency or control than IO. It is still relatively simple but allows you to use ByteBuffers. e.g. ByteBuffers support little endian.
A common option is to use non-blocking NIO with Selectors. Much of the complexity this introduces can be handled by frameworks such as Netty or Mina. I suggest you use such a library if you need non-blocking IO e.g. because you have thousands of concurrent connections per server. IMHO You have thousands of connections, you should consider having more servers unless what each connection does is pretty trivial. AFAIK google go for more servers rather thousands of users per server.
The more extreme option is to use NIO2. This is even more complex and lengthy it write than non-blocking NIO. I don't know of any frameworks which support this well. i.e. it is actually faster when you do. AFAIK It appears this is worth using if you have Infiniband (which is what it was designed to support) but perhaps not worth using if you have Ethernet.
If you want non-blocking IO, NIO is not the better choice—it's the only choice in Java. Keep in mind that people still use the old IO regularly because it is way simpler to code against. NIO API is quite raw and is more of an enabling low-level technology than a client-side API. I suggest using NIO through an API that provides a simpler interface to the problems you want to solve using non-blocking IO.
A little late, but personally, I use NIO even for the regular "everyday" file handling. So, I use things like:
1. if(Files.notExists(path)) { }
2. Files.createDirectory(path);
3. Files.newInputStream(path) targetPath.resolve("somefile.txt");
4. Files.newBufferedWriter(path, charset);
5. DirectoryStream<Path> directoryStream = Files.newDirectoryStream(path);
and it works fine for me. I prefer Path instead of the old File because of methods like relativize or resolveSibling.
Doesn't strike me as more complicated than IO.
You would only use NIO if you can justify the inevitable complexity that it introduces. If you do not have any guidance in terms of the expected load, and also in terms of whether your product / project has the resources to maintain the relevant code, then your should err on the side of caution and use IO.
To give my answer some weight, I have just spent three months maintaining and bug fixing an integration layer where raw Java NIO (i.e. no overarching framework was used) was used. The design was based, in essence, on client threads adding messages to a queue and a small number of worker threads performing their NIO magic and then passing replies back to client threads in an event-based manner. In retrospect, I cannot justify the original decision to use NIO, since it became a distraction that ate significant amounts of time that should have been spent on higher level business logic.
You can use any of this, unless you are going to create "super fast" server.
Of course a good approach here is to use nio, since it's new and modern way to write multi-client servers for high throughput tasks.
Some advantages of the NIO.2 API over the legacy java.io.File class for working with files:
Supports file system–dependent attributes.
Allows you to traverse a directory tree directly.
Supports symbolic links.
For specific use cases and more details, you can see this article
Traditional IO is easy and simplified code, NIO is more complicated but more flexible.
In my case i prefer use IO for small streaming and NIO for large streaming but nio is really more complex
with NIO i have to create an entire package to manage it instead io package that i directly use snippet
At the moment I have a solution that uses ZeroMQ to exchange protocol buffer payloads.
The protocol buffer method of serialization is bound to stay as it is, but I can replace ZMQ with a more convenient option.
The things I am not happy about in ZMQ are:
It uses JNI on the Java side,and I've been bitten before by JNI, in complex, multi thread scenarios. I try to eliminate it whenever I can.
I don't need queuing, I just need rpc.
My requirements (which are mostly covered by ZeroMQ) are:
Support for 32/64 bit *nix, Windows, MacOS.
Support for Java, C++ and C# primarily, and Python, Ruby etc. would be nice.
Language support must be provided by native implementations in the language, not via wrapping native code.
High performance.
Non Viral license, no GPL, AGPL etc.
I've been thinking about using Thrift as the transport layer over TCP (I guess it supports that) with protocol buffers payloads, if its Java implementation for messaging is not using JNI.
What options can you think of other than ZMQ for this setup?
You should probably have a look at Netty. It's a high performance Java NIO server framework with built-in support for Protocol Buffer which is released under the terms of the Apache License. The framework is well documented and some examples show how to prototype protocols with Protocol Buffers.
Have you considered something like Storm or Spread?
The original question was asked about a year after JeroMQ was put onto github. It is the pure-java implementation of ZeroMQ. It has seen constant development throughout the intervening years and seems to be comparable in speed to the C-implementation.
While searching the web for concurrency in jvm I found questions about searching Non-blocking IO library for Scala / Java.
What is the problem about? If I want to send something to file / socket I can launch separate thread which make the job.
I know there could be problem using event based threads - because whole system could be blocked. But does it reference to JVM/ Scala?
ADDED:
Please correct me if I'm wrong:
I think that when you need to call some IO function in asynchronous way it need to go into separate process or system (heavy) thread. Am I right?
So - all the questions about solving this kind of thing in common languages goes into creating and managing separate process or threads. So the only facilitate from the language is to create some pool of threads which will be assigned to IO operations in async.
So my hypotheses is.
Sentence: Language X is better then Y because calling async IO operation dosen't block the virtual machine is false because in every language that support system threads there is possibility to manage NIO, the only difference is that language X has better support for this through builtin libraries / language features.
Is this hypothese Truth?
Can some language achieve NIO without os system support? (through processes / threads)
Scala has a bunch of tools for concurrency, and NIO has a few tools for non-blocking IO. So, it should come as no surprise that there are a lot of great libraries that help connect the dots:
Finagle
... a library for building
asynchronous RPC servers and clients
in Java, Scala, or any JVM language.
Built atop Netty, Finagle provides a
rich set of tools that are protocol
independent.
Akka is a pretty nice, featureful actors/concurrency/services package which also uses Netty for their built-in remoting functionality
Naggati2 is another one from Twitter, also built on Netty, not sure if it's being superseded by Finagle though.
Here is an interesting recent blog post that may help you: http://jim-mcbeath.blogspot.com/2011/03/java-nio-and-scala-continuations.html
What is the "official" Java API for client/server or P2P communication? Java RMI? Some other networking API??
Is this official networking API the standard for both SE and EE?
I'm sure the answer is very context-specific, so let's take a look at a few instances:
You have 2 swing clients installed on 2 machines and connected to the same network (or the Internet), and you want either of them to send the other a primitive, such as the integer 4, or some POJO, like a "Widget" object
Same as #1 above, but between a Swing client and a fully-compliant Java EE back-end (implementing managed beans, app servers, the whole nine yards)
I don't have a specific application in mind, I'm just wondering what are the "norms" for client-client and client-server communication in the world of Java.
If being bound by Java isn't a problem, RMI is a pretty abstracted solution when it comes to the client and server solution "exchanging" data (especially when the data is Java classes which might be difficult/too much effort to represent as textual data). Just make sure your object implements Serializable and almost anything can be transmitted over the wire.
If this doesn't fit your bill and you want to drop down the raw networking stuff, the client-server socket framework Netty is a pretty good choice.
There's no such thing as the most official networking API in J2SE, all J2SE APIs are official in the sense they are supported by Sun (now Oracle).
That said, you should choose your API based on following criteria:
Do you (or your team) know how to use particular API;
How simple/complex is this API to use;
What throughput are you aiming for? For performance-sensitive applications you may be forced to use binary protocol. For the rest of cases, you can use text-based protocol.
For example, between two clients simple text-based protocol will suffice for passing POJOs, for example using Apache MINA or Google protocol buffers.
This will work between client and server as well.
Response to Zac's questions in comment:
Binary protocols performance gain comes from the fact you don't need to convert everything to text form and back -- you just can pass binary presentation of you application memory with minimal changes, like, in case of BSD Sockets API, converting from host byte-order to network byte-order. Unfortunately, I don't know details about how RMI/Java serialization processes objects, but I'm sure, it still much faster than passing all data in readable form;
Yes, MINA and protocol buffers have Java APIs. They just not part of Java SE bundle, you have to download them separately. By the way, MINA can use both binary and readable serialization, depending on how you use it.
You should define notion of 'good' somehow, for example, answering to questions I mentioned above. If you want to use objects over network, use RMI. If you don't, Netty or MINA will suffice, whatever you'll find easier to master.
For P2P, Sun at one point pushed JXTA pretty hard.
I wouldn't dare to use RMI for P2P communication.
rmi is pretty much the standard java to java protocol. it's built in and very simple to use. most j2ee backends also communicate using rmi, although that's not the only possibility.
J2SE the most common is probably RMI or raw sockets.
J2EE uses a messaging bus that everyone (servers and clients) subscribes to which is quite different from rmi style solutions (although at the lowest level an implementation may still rely on RMI). It helps automate redundancy and failover. If you need this functionality I believe it can be used in SE as well.
I haven't used J2EE for quite a while now, so this may have changed, but I doubt it. The messaging system was a core component of J2EE.
The Java NIO Socket Framework supposedly hides the dirty details of non-blocking IO from developers, allowing them to build highly scalable applications, which can handle over 10000 incoming and outgoing sockets using only one thread.
Are non blocking IOs still a pain with the typical version of Java 2 SE/EE?
Is this framework still necessary and useful?
Thanks for your time.
Well, NIO creates an abstraction over some of the details, certainly. Non-blocking IO is still a pain to get your head around (at least, I find it is) but at least it's feasible. (Personally I prefer the .NET style of asynchronous IO, but that's a different matter.)
I usually use blocking IO: for most tasks, this is all I require and I wouldn't gain significantly by using non-blocking IO. In some cases (such as the one you mentioned) non-blocking IO is really the only way forward if you want to keep your thread down.
I recommend that you learn about it, play with it, and then use judgement to decide when to use it in production code. I wouldn't suggest starting to use it everywhere...
Yes, NIO is very useful. NIO is also a bit hard to work with.
Depending on your needs you could consider using frameworks that wrap NIO, like grizzly or mina. Grizzly is the networking part of glassfish appserver from sun Oracle.
Mina is a network application framework from Apache.org.
Personally I prefer grizzly but that's just me.