Develop Reference

How to write MappedByteBuffer to socket outputstream (server to client) with no copy

In Java code, I have a region of file mapped using MappedByteBuffer and I need to send this to the client (write to Outputstream). I need to make sure that while sending/writing to socket, it does not create any copy due to memory constraints. How can I achieve this? Will Bytebuffer.array() serve this purpose?
Sharing the code. Note: FileChannel is read-only and I need to send ByteBuffer data as it is.
private void writeData(Socket clientSocket, MappedByteBuffer byteBuffer){
Path path = Paths.get(myfile);
MappedByteBuffer memoryMappedBuffer = null;
try (FileChannel fChannel = FileChannel.open(path, StandardOpenOption.READ)) {
memoryMappedBuffer = fChannel.map(FileChannel.MapMode.READ_ONLY, location, size);
}catch(){
//How can i write memoryMappedBuffer to socket outputStream without copying data...? like
clientSocket.getOutputStream().write(memoryMappedBuffer .array());
}

If you are creating the socket yourself, you can use SocketChannel.open() instead, and use write(ByteBuffer). It manages the socket internally.
InetSocketAddress address = ...
SocketChannel channel = SocketChannel.open(address);
channel.write(memoryMappedBuffer);
// ...
channel.close(); // Closes the connection
If you have a pre-existing socket, you can create a Channel from the socket's output stream. However this allocates a (reused) temporary buffer.
WritableByteChannel channel = Channels.newChannel(clientSocket.getOutputStream());
channel.write(memoryMappedBuffer);
// ...
channel.close();

Based on your description you want to write to a socket the content of a file, or a slice of the file, slice that it is mapped to the virtual address space of your java process.
The memory-mapped files is used in general to share memory between multiple processes or to minimize the I/O operations on disk when your process is writing and reading data to/from the same files (a concrete example is Kafka which uses this procedure).
In your case, when you write the data to a socket, that socket has a buffer (regardless if it's blocking or non-blocking). When the buffer is full you will not be able to write, until the receiver acknowledge the data, and the buffer is cleared. Now, if the receiver is slow, you will remain with that portion of the file loaded to your main memory for a long time which can affect the performance of your server (I suppose you will not have a single consumer/client for your server).
One good and efficient solution is to use pipe streaming which sends data from your server to a consumer (in this case a socket) in a producer/consumer way. By default the PipeInputStream uses a buffer of 1024 bytes (you can increase it), meaning that only 1024 bytes will be kept in the memory at one moment in time for a specific execution thread. If your process has 500 clients, then you will consume only 500*1024 bytes = 512Kb. If one reader is slow, then that producer will be slow also without putting pressure on your process memory.
If all you have to do is to write the content of various files to sockets, I don't know how using memory-mapped files can helps you.

java Non blocking network programming time out issue

I am writing ftp server in java using NIO non-blocking technology.
I want to prevent user from connect to my server and then do nothing.
Here is my code snippet:
ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
SocketChannel socketChannel = serverChannel.accept();
socketChannel.socket().setSoTimeout(3000);
socketChannel.configureBlocking(false);
....................
It does not works. Um... is it possible to throw an exception when the user do nothing (e.g. say 15 minutes)?
thank you very much

socketChannel.socket().setSoTimeout(3000);
You've done it. But then you also put the channel into non-blocking mode, which prevents you getting timeout exceptions. If you're using non-blocking mode, and therefore presumably also select(), you have to manage the timeout yourself.

SocketChannel read() behaviour - short reads

The ServerSocketChannel is used this way:
ServerSocketChannel srv = ServerSocketChannel.open();
srv.socket().bind(new java.net.InetSocketAddress(8112));
SocketChannel client = srv.accept();
When a connection is received, data is read this way:
ByteBuffer data = ByteBuffer.allocate(2000);
data.order(ByteOrder.LITTLE_ENDIAN);
client.read(data);
logger.debug("Position: {} bytes read!", data.position());
It prints:
Position: 16 bytes read!
Why isn't the SocketChannel blocking until the buffer is filled?
From the ServerSocketChannel.accept() API (Java 7):
The socket channel returned by this method, if any, will be in
blocking mode regardless of the blocking mode of this channel.
Does the write(ByteBuffer buffer) of the SocketChannel block? How do I test that anyway?
Thank you for your time!

Blocking mode means that it blocks until any data is received. It doesn't have to be an entire buffer full.
If you want to make sure you've received an entire bufferful of data, you should read() in a loop until you've filled up your buffer.

Timeout for SocketChannel doesn't work

I want to use a SocketChannel and to have a timeout for its read/write methods. I've tried to set a timeout for the Socket that owns my SocketChannel like this:
channel.socket().setSoTimeout(TIMEOUT);
but that doesn't work. Is there any other solution?

According to this article, SocketChannel will not timeout for its read operation but you can get this effect from reading from the channel in another way.
SocketChannel socketChannel;
socketChannel.socket().setSocketTimeout(500);
InputStream inStream = socketChannel.socket().getInputStream();
ReadableByteChannel wrappedChannel = Channels.newChannel(inStream);
reading from the wrappedChannel will timeout according to the socketTimeOut you have set.

If you are familiar with using Java Selector, you can emulate socket timeout yourself using selector. It is helpful to see sun.nio.ch.SocketAdaptor.
It should be careful to use Thread.interrupt().
SocketChannel is InterruptibleChannel. As you read the description of InterruptibleChannel, Thread.interrupt() causes to close SocketChannel.
If you want to use SocketChannel after timeout, you cannot use the InterruptibleChannel feature.

You could also consider making your channel non-blockable and just using System.currentTimeMillis().

java/groovy socket write timeout

I have a simple badly behaved server (written in Groovy)
ServerSocket ss = new ServerSocket(8889);
Socket s = ss.accept()
Thread.sleep(1000000)
And a client who I want to have timeout (since the server is not consuming it's input)
Socket s = new Socket("192.168.0.106", 8889)
s.setSoTimeout(100);
s.getOutputStream.write( new byte[1000000] );
However, this client blocks forever. How do I get the client to timeout?
THANKS!!

You could spawn the client in it's own thread and spin lock/wait(timeout long) on it to return. Possibly using a Future object to get the return value if the Socket is successful.
I do believe that the SO_TIMEOUT setting for a Socket only effects the read(..) calls from the socket, not the write.
You might try using a SocketChannel (rather then Stream) and spawn another thread that also has a handle to that Channel. The other thread can asynchronously close that channel after a certain timeout of it is blocked.

The socket timeout is at the TCP level, not at the application level. The source machine TCP is buffering the data to be sent and the target machine network stack is acknowledging the data received, so there's no timeout. Also, different TCP/IP implementations handle these timeouts differently. Take a look at what's going on on the wire with tcpdump (or wireshark if you are so unfortunate :) What you need is application level ACK, i.e. you need to define the protocol between client and the server. I can't comment on Java packages (you probably want to look at nio), but receive timeout on that ACK would usually be handled with poll/select.

There is no way to get the timeout, but you can always spawn a thread that closes the connection if the write hasn't finished.