I have been experiencing this issue for a while now. A quick explanation of the system:
A simple application will read in data over a tcp connection. The application uses Socketchannel object to establish a data communication line. The connection to hardware is established, and the application processes roughly between 400 - 700 packets before throwing the error:
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
at sun.nio.ch.IOUtil.read(IOUtil.java:186)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
I suspect it has to do with the operating system (windows 7) buffer, although I have tried a few work arounds (increasing JVM heap size, and creating additional registry variables for TCP settings), but with no luck.
Inspecting packet data using wireshark, the following information can been seen just as the error occurs
[TCP Window Full] 5001 > 49995 [PSH, ACK] Seq=123177 Ack=1 Win=200 Len=176
[TCP ZeroWindow] 49995 > 5001 [ACK] Seq=1 Ack=123353 Win=0 Len=0
Which is then followed by a series of
[TCP Dup ACK 144#1] 5001 > 49995 [ACK] Seq=123353 Ack=1 Win=200 Len=0
until the connection is terminated.
I am hoping perhaps someone has had a similar issue in the past,or could offer some guidance on something I may be overlooking.
Here is sample code where data is read, and the error is thrown.
try{
do{
socketChannel.read(syncBuf);
start = syncBuf.get(0);
int b = start;
System.out.println("start byte is " + b);
syncBuf.clear();
}while(start != INIT_BYTE);
}catch(Exception e){
e.printStackTrace();
}
packetBuf.order(ByteOrder.LITTLE_ENDIAN);
packetBuf.clear();
try{
while(socketChannel.read(packetBuf) != 206){
nrBytesRead += socketChannel.read(packetBuf);
}
}catch(Exception e){
ApplicationLogger.log("Error at INIT_BYTE loop ", "ERROR");
}
Many thanks.
I think you have a problem here:
while(socketChannel.read(packetBuf) != 206)
{
nrBytesRead += socketChannel.read(packetBuf);
}
socketChannel.read(packetBuf) will return the number read this time, not the total, and I guess that's what 206 is, the total read.
You probably want to do that:
do
{
nrBytesRead += socketChannel.read(packetBuf);
}
while (nrBytesRead != 206)
Related
Two parts:
InetAddress ia = Inet4Address.getByAddress(new byte[]{(byte)192,(byte)168,(byte)10,(byte)100});
Socket socket = new Socket();
InetSocketAddress isa = new InetSocketAddress("82.146.58.81", 5277);
socket.bind(new InetSocketAddress(ia, 0));
System.out.println("local_address:" + socket.getLocalSocketAddress());
socket.connect(isa, 1000);
Thread.sleep(1000);
socket.close();
System.out.println("done_1");
InetAddress ia_2 = Inet4Address.getByAddress(new byte[]{(byte)172,(byte)20,(byte)10,(byte)2});
Socket socket_2 = new Socket();
InetSocketAddress isa2 = new InetSocketAddress("82.146.58.81", 5277);
socket_2.bind(new InetSocketAddress(ia_2, 0));
System.out.println("local_address:" + socket_2.getLocalSocketAddress());
socket_2.connect(isa2, 30000);
Thread.sleep(1000);
socket_2.close();
System.out.println("done_2");
First part of code worked, second not - java.net.SocketTimeoutException: connect timed out.
I am trying to communicate with the host "82.146.58.81" through different interfaces.
But only the part that has the default route is successful. Why?
Update: I am worked by Ubuntu 17.10, seems the same problem in Armbian too. Problem not reproducible under Windows7
Update: Wrong OUT interface:
[12979.049742] IN= OUT=enp4s0 SRC=172.20.10.2 DST=82.146.58.81 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=25967 DF PROTO=TCP SPT=48985 DPT=5277 WINDOW=29200 RES=0x00 SYN URGP=0
kostas#work:~$ ifconfig
enp0s16f0u1c4i2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.20.10.2 netmask 255.255.255.240 broadcast 172.20.10.15
inet6 fe80::17c7:4582:897b:11d5 prefixlen 64 scopeid 0x20<link>
ether 86:38:35:c8:f5:39 txqueuelen 1000 (Ethernet)
RX packets 575 bytes 117085 (117.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 686 bytes 82681 (82.6 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.10.100 netmask 255.255.255.0 broadcast 192.168.10.255
inet6 fe80::80b9:8cf3:92c6:4aef prefixlen 64 scopeid 0x20<link>
ether 00:e0:66:ae:2a:de txqueuelen 1000 (Ethernet)
RX packets 100455 bytes 65543315 (65.5 MB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 75513 bytes 15658830 (15.6 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
kostas#work:~$ route
Таблица маршутизации ядра протокола IP
Destination Gateway Genmask Flags Metric Ref Use Iface
default unknown 0.0.0.0 UG 100 0 0 enp4s0
default gateway 0.0.0.0 UG 101 0 0 enp0s16f0u1c4i2
kostas#work:~$ ping mail.ru -I enp0s16f0u1c4i2
PING mail.ru (217.69.139.202) from 172.20.10.2 enp0s16f0u1c4i2: 56(84) bytes of data.
64 bytes from mail.ru (217.69.139.202): icmp_seq=1 ttl=51 time=508 ms
64 bytes from mail.ru (217.69.139.202): icmp_seq=2 ttl=51 time=187 ms
"However, many OSes do not honour this "hint" and will use the routing table anyway."
Answer here
Im searched a lot about this topic.
UDP is not reliable
UDP can't send packets higher than 64kbs or 65507 bytes without headers
the MTU normally is 1500
sysctl.conf
net.core.rmem_max = 52508000
net.core.rmem_default = 26265600
net.ipv4.udp_mem = 102400 26265600 52508000
net.core.netdev_max_backlog = 65536 net.ipv4.route.flush = 1
I made test in Windows for an escenary where i try to send 100msg/sec # 256 kbps in 300 sec, with no packet loss through Localhost.
Tried Linux throgh Localhost fragment 256kbs in 4 msg of 65507 bytes, with 125000~ msg published 120000~ Received 4% packet loss.
Trying between 2 subnet i get horrible results , publishing 125000~ and receiving 40000~.
My machine: 1 GB Ethernet , 32 core , 32 gb ram Linux Kernel 2.6.32-358.el6.x86_64
My app is designed with 10 threads sending to 10 ports from an specific IP.
My UDP connector in Mule :
<udp:connector name="udpConnector" timeout="0" validateConnections="false" receiveBufferSize="26265600" sendBufferSize="26275600" broadcast="true" keepSendSocketOpen="true" doc:name="udpConnector"/>
Maybe i forgot settings or my Mule UDP pararms are wrong, but i can't explain this with a nice machine.
Someone can iluminate me? A lot of thnx, and TCP is not an option ^^
//sendData = new byte[65507];
tim = 300;
delay = tim * 1000;
setmsgSec(40);
do{
for(int i=0; i<=msgSec ;i++)
{
ds.send(dp[3]);
cont++;
setCont(cont);
setRate(cont);
setAvgBytes(sendata[3].length);
}
Thread.sleep(1000);
tim = tim - 1;
delay = delay - 1000;
}while (delay != 0);
setCont(cont);
Problem solved.
Solution: put the same values for buffers on each machine, for 1GB Ethernet, 32 core & 32G RAM and burst of 200 messages 256kbps this values are enough.
kernel.threads-max = 2048
net.core.optmem_max = 16777216
net.core.rmem_default = 2620280
net.core.rmem_max = 16777216
net.core.wmem_default = 2620280
net.core.wmem_max = 16777216**
I Use a log4j.properties with low cost to performance:
log4j.rootLogger=INFO, default.out
# System.out.println appender for all classes
log4j.appender.default.out=org.apache.log4j.ConsoleAppender
log4j.appender.default.out.threshold=INFO
log4j.appender.default.out.layout=org.apache.log4j.PatternLayout
log4j.appender.default.out.layout.ConversionPattern=%m%n
I think I am very close to getting my Java server app to talk to a browser page via WebRTC, but I can't quite get it to work. I feel like I am missing something small, so I hope that someone here might have a suggestion.
I took a close look at the WebRTC examples - the Java unit test (org.webrtc.PeerConnectionTest), and the example Android app (trunk/talk/examples/android). Based on what I learned, I put together a java app that uses WebSockets for signalling and attempts to send a video stream to Chrome.
The problem is that there is no video in the browser, even though all of my code (both Javascript and Java) is executed in the order I expect it to, hitting all the correct logging statements. There is some suspicious output in the console log from the native libjingle code, but I am not sure what to make of it. I have highlighted suspicious lines in the log with '>>' below. For example, it seems that the video port allocators are destroyed shortly after being created, so something is obviously wrong. Also, "Changing video state, recv=1 send=0" seems incorrect as well, since the Java side should be sending the video, rather than receiving.... Perhaps I am misusing the OfferToReceiveVideo option?
If you look at the logs below, you'll see that the WebSocket communication with the browser is working perfectly, and that I am able to successfully send the SDP Offer to the browser and receive an SDP Answer from the browser. Setting the Local and Remote Descriptions on the PeerConnections seems to work properly as well. The HTML5 video element gets the source set to a BLOB url, as it should. So, what could I be missing? Do I need to do anything with ICE candidates, even though my client and server are on the same machine right now?
Any advice would be greatly appreciated!
SDP Messages (from Chrome's Javascript Console)
1.134: Java Offer:
v=0
o=- 5893945934600346864 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE video
a=msid-semantic: WMS JavaMediaStream
m=video 1 RTP/SAVPF 100 116 117
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:dJxTlMlXy7uASrDU
a=ice-pwd:r8BRkXVnc4dqCABUDhuRjpp7
a=ice-options:google-ice
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=sendrecv
a=rtcp-mux
a=crypto:0 AES_CM_128_HMAC_SHA1_80 inline:yq6wOHhk/QfsWuh+1oOEqfB4GjKZzz8XfQnGCDP3
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=ssrc:3720473526 cname:nul6R21KmwAms3Ge
a=ssrc:3720473526 msid:JavaMediaStream JavaMediaStream_v0
a=ssrc:3720473526 mslabel:JavaMediaStream
a=ssrc:3720473526 label:JavaMediaStream_v0
1.149: Received remote stream
1.150: Browsers Answer:
v=0
o=- 4261396844048664099 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE video
a=msid-semantic: WMS
m=video 1 RTP/SAVPF 100 116 117
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:quzQNsX+ZlUWUQqV
a=ice-pwd:y5A0+7sM8P88AatBLd1fdd5G
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=recvonly
a=rtcp-mux
a=crypto:0 AES_CM_128_HMAC_SHA1_80 inline:WClNA69OfpjdJy3Bv4ujejk/IYnn4DW8kjrB18xP
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
This seems ok to me. Java's offer includes my video stream.
Native code logging (libjingle)
(Suspicious lines marked with '>>')
Camera '/dev/video0' started with format YUY2 640x480x30, elapsed time 59 ms
Ignored line: c=IN IP4 0.0.0.0
NACK enabled for channel 0
NACK enabled for channel 0
Created channel for video
Jingle:Channel[video|1|__]: NULL DTLS identity supplied. Not doing DTLS
Jingle:Channel[video|2|__]: NULL DTLS identity supplied. Not doing DTLS
Session:5893945934600346864 Old state:STATE_INIT New state:STATE_SENTINITIATE Type:urn:xmpp:jingle:apps:rtp:1 Transport:http://www.google.com/transport/p2p
Setting local video description
AddSendStream {id:JavaMediaStream_v0;ssrcs:[3720473526];ssrc_groups:;cname:nul6R21KmwAms3Ge;sync_label:JavaMediaStream}
Add send ssrc: 3720473526
>> Warning(webrtcvideoengine.cc:2704): SetReceiverBufferingMode(0, 0) failed, err=12606
Changing video state, recv=0 send=0
Transport: video, allocating candidates
Transport: video, allocating candidates
Jingle:Net[eth0:192.168.0.0/24]: Allocation Phase=Udp
Jingle:Port[:1:0::Net[eth0:192.168.0.0/24]]: Port created
Adding allocated port for video
Jingle:Port[video:1:0::Net[eth0:192.168.0.0/24]]: Added port to allocator
Jingle:Net[tun0:192.168.128.6/32]: Allocation Phase=Udp
Jingle:Port[:1:0::Net[tun0:192.168.128.6/32]]: Port created
Adding allocated port for video
Jingle:Port[video:1:0::Net[tun0:192.168.128.6/32]]: Added port to allocator
Ignored line: c=IN IP4 0.0.0.0
Warning(webrtcvideoengine.cc:2309): GetStats: sender information not ready.
Jingle:Channel[video|1|__]: Other side didn't support DTLS.
Jingle:Channel[video|2|__]: Other side didn't support DTLS.
Enabling BUNDLE, bundling onto transport: video
Channel enabled
>> Changing video state, recv=1 send=0
Session:5893945934600346864 Old state:STATE_SENTINITIATE New state:STATE_RECEIVEDACCEPT Type:urn:xmpp:jingle:apps:rtp:1 Transport:http://www.google.com/transport/p2p
Setting remote video description
Hybrid NACK/FEC enabled for channel 0
Hybrid NACK/FEC enabled for channel 0
SetSendCodecs() : selected video codec VP8/1280x720x30fps#2000kbps (min=50kbps, start=300kbps)
Video max quantization: 56
VP8 number of temporal layers: 1
VP8 options : picture loss indication = 0, feedback mode = 0, complexity = normal, resilience = off, denoising = 0, error concealment = 0, automatic resize = 0, frame dropping = 1, key frame interval = 3000
WARNING: no real random source present!
SRTP activated with negotiated parameters: send cipher_suite AES_CM_128_HMAC_SHA1_80 recv cipher_suite AES_CM_128_HMAC_SHA1_80
Changing video state, recv=1 send=0
Session:5893945934600346864 Old state:STATE_RECEIVEDACCEPT New state:STATE_INPROGRESS Type:urn:xmpp:jingle:apps:rtp:1 Transport:http://www.google.com/transport/p2p
Jingle:Net[eth0:192.168.0.0/24]: Allocation Phase=Relay
Jingle:Net[tun0:192.168.128.6/32]: Allocation Phase=Relay
Jingle:Net[eth0:192.168.0.0/24]: Allocation Phase=Tcp
Jingle:Port[:1:0:local:Net[eth0:192.168.0.0/24]]: Port created
Adding allocated port for video
Jingle:Port[video:1:0:local:Net[eth0:192.168.0.0/24]]: Added port to allocator
Jingle:Net[tun0:192.168.128.6/32]: Allocation Phase=Tcp
Jingle:Port[:1:0:local:Net[tun0:192.168.128.6/32]]: Port created
Adding allocated port for video
Jingle:Port[video:1:0:local:Net[tun0:192.168.128.6/32]]: Added port to allocator
Jingle:Net[eth0:192.168.0.0/24]: Allocation Phase=SslTcp
Jingle:Net[tun0:192.168.128.6/32]: Allocation Phase=SslTcp
All candidates gathered for video:1:0
Transport: video, component 1 allocation complete
Transport: video allocation complete
Candidate gathering is complete.
Capture delay changed to 120 ms
Captured frame size 640x480. Expected format YUY2 640x480x30
Capture size changed : selected video codec VP8/640x480x30fps#2000kbps (min=50kbps, start=300kbps)
Video max quantization: 56
VP8 number of temporal layers: 1
VP8 options : picture loss indication = 0, feedback mode = 0, complexity = normal, resilience = off, denoising = 0, error concealment = 0, automatic resize = 1, frame dropping = 1, key frame interval = 3000
VAdapt Frame: 0 / 300 Changes: 0 Input: 640x480 Scale: 1 Output: 640x480 Changed: false
>> Jingle:Port[video:1:0::Net[eth0:192.168.0.0/24]]: Port deleted
>> Jingle:Port[video:1:0::Net[eth0:192.168.0.0/24]]: Removed port from allocator (3 remaining)
Removed port from p2p socket: 3 remaining
Jingle:Port[video:1:0::Net[tun0:192.168.128.6/32]]: Port deleted
Jingle:Port[video:1:0::Net[tun0:192.168.128.6/32]]: Removed port from allocator (2 remaining)
Removed port from p2p socket: 2 remaining
>> Jingle:Port[video:1:0:local:Net[eth0:192.168.0.0/24]]: Port deleted
>> Jingle:Port[video:1:0:local:Net[eth0:192.168.0.0/24]]: Removed port from allocator (1 remaining)
Removed port from p2p socket: 1 remaining
Jingle:Port[video:1:0:local:Net[tun0:192.168.128.6/32]]: Port deleted
Jingle:Port[video:1:0:local:Net[tun0:192.168.128.6/32]]: Removed port from allocator (0 remaining)
Removed port from p2p socket: 0 remaining
HTML
<html lang="en">
<head>
<title>Web Socket Signalling</title>
<link rel="stylesheet" href="css/socket.css">
<script src="js/socket.js"></script>
</head>
<body>
<h2>Repsonse from Server</h2>
<textarea id="responseText"></textarea>
<h2>Video</h2>
<video id="remoteVideo" autoplay></video>
</body>
</html>
Javascript
(function() {
var remotePeerConnection;
var sdpConstraints = {
'mandatory' : {
'OfferToReceiveAudio' : false,
'OfferToReceiveVideo' : true
}
};
var Sock = function() {
var socket;
if (!window.WebSocket) {
window.WebSocket = window.MozWebSocket;
}
if (window.WebSocket) {
socket = new WebSocket("ws://localhost:8080/websocket");
socket.onopen = onopen;
socket.onmessage = onmessage;
socket.onclose = onclose;
} else {
alert("Your browser does not support Web Socket.");
}
function onopen(event) {
getTextAreaElement().value = "Web Socket opened!";
}
function onmessage(event) {
appendTextArea(event.data);
sdpOffer = new RTCSessionDescription(JSON.parse(event.data));
remotePeerConnection = new webkitRTCPeerConnection(null);
remotePeerConnection.onaddstream = gotRemoteStream;
trace("Java Offer: \n" + sdpOffer.sdp);
remotePeerConnection.setRemoteDescription(sdpOffer);
remotePeerConnection.createAnswer(gotRemoteDescription, onCreateSessionDescriptionError, sdpConstraints);
}
function onCreateSessionDescriptionError(error) {
console.log('Failed to create session description: '
+ error.toString());
}
function gotRemoteDescription(answer) {
remotePeerConnection.setLocalDescription(answer);
trace("Browser's Answer: \n" + answer.sdp);
socket.send(JSON.stringify(answer));
}
function gotRemoteStream(event) {
var remoteVideo = document.getElementById("remoteVideo");
remoteVideo.src = URL.createObjectURL(event.stream);
trace("Received remote stream");
}
function onclose(event) {
appendTextArea("Web Socket closed");
}
function appendTextArea(newData) {
var el = getTextAreaElement();
el.value = el.value + '\n' + newData;
}
function getTextAreaElement() {
return document.getElementById('responseText');
}
function trace(text) {
console.log((performance.now() / 1000).toFixed(3) + ": " + text);
}
}
window.addEventListener('load', function() {
new Sock();
}, false);
})();
Java Server
public class PeerConnectionManager {
/**
* Called when the WebSocket handshake is completed
*/
public void createOffer() {
peerConnection = factory.createPeerConnection(
new ArrayList<PeerConnection.IceServer>(),
new MediaConstraints(),
new PeerConnectionObserverImpl());
// Get the video source
videoSource = factory.createVideoSource(VideoCapturer.create(""), new MediaConstraints());
// Create a MediaStream with one video track
MediaStream lMS = factory.createLocalMediaStream("JavaMediaStream");
VideoTrack videoTrack = factory.createVideoTrack("JavaMediaStream_v0", videoSource);
videoTrack.addRenderer(new VideoRenderer(new VideoRendererObserverImpl()));
lMS.addTrack(videoTrack);
peerConnection.addStream(lMS, new MediaConstraints());
// We don't want to receive anything
MediaConstraints sdpConstraints = new MediaConstraints();
sdpConstraints.mandatory.add(new MediaConstraints.KeyValuePair(
"OfferToReceiveAudio", "false"));
sdpConstraints.mandatory.add(new MediaConstraints.KeyValuePair(
"OfferToReceiveVideo", "false"));
// Get the Offer SDP
SdpObserverImpl sdpOfferObserver = new SdpObserverImpl();
peerConnection.createOffer(sdpOfferObserver, sdpConstraints);
SessionDescription offerSdp = sdpOfferObserver.getSdp();
// Set local SDP, don't care for any callbacks
peerConnection.setLocalDescription(new SdpObserverImpl(), offerSdp);
// Serialize Offer and send to the Browser via a WebSocket
JSONObject offerSdpJson = new JSONObject();
offerSdpJson.put("sdp", offerSdp.description);
offerSdpJson.put("type", offerSdp.type.canonicalForm());
webSocketContext.channel().writeAndFlush(
new TextWebSocketFrame(offerSdpJson.toString()));
}
/**
* Called when an SDP Answer arrives via the WebSocket
*/
public void setRemoteDescription(SessionDescription answer) {
peerConnection.setRemoteDescription( new SdpObserverImpl(), answer);
}
}
Ugh. Never mind. Sorry for the stupid question.
The missing part was the exchange of ICE candidates between the browser and Java server. Now that I added the code to do ICE negotiation via my WebSocket, everything works fine!
I have a java app program run on centos 6.3 and tomcat 7 as the app container, currently I meet one error : java.io.socketexception Maximum number of datagram sockets reached
we use MulticastSocket class to send message. when this error happened, I check the current server UDP socket count with command: ss -s
Total: 212 (kernel 248)
TCP: 70 (estab 15, closed 44, orphaned 0, synrecv 0, timewait 40/0), ports 22
Transport Total IP IPv6
* 248 - -
RAW 0 0 0
UDP 40 40 0
TCP 26 26 0
INET 66 66 0
FRAG 0 0 0
and I also check the
ulimits -n
The default setting is 32768, seem UDP socket count not exceed max count.
Any ideas for this error?
we use MulticastSocket class to send message.
Why? You only need a MulticastSocket to receive multicasts.
Obviously you are leaking MulticastSockets. Presumably you are creating a new one per message and never closing it.
I'm new to Scala so the question may be quite simple, though I have spent some time trying to resolve it. I have a simple Scala TCP server (no actors, single thread):
import java.io._
import java.net._
object Application {
def readSocket(socket: Socket): String = {
val bufferedReader = new BufferedReader(new InputStreamReader(socket.getInputStream))
var request = ""
var line = ""
do {
line = bufferedReader.readLine()
if (line == null) {
println("Stream terminated")
return request
}
request += line + "\n"
} while (line != "")
request
}
def writeSocket(socket: Socket, string: String) {
val out: PrintWriter = new PrintWriter(new OutputStreamWriter(socket.getOutputStream))
out.println(string)
out.flush()
}
def main(args: Array[String]) {
val port = 8000
val serverSocket = new ServerSocket(port)
while (true) {
val socket = serverSocket.accept()
readSocket(socket)
writeSocket(socket, "HTTP/1.1 200 OK\r\n\r\nOK")
socket.close()
}
}
}
The server listens on localhost:8000 for incomming requests and sends HTTP response with single OK word in the body. Then I run Apache Benchmark like this:
ab -c 1000 -n 10000 http://localhost:8000/
which works nicely for the first time. The second time I start ab it hangs producing the following output in netstat -a | grep 8000:
....
tcp 0 0 localhost.localdo:43709 localhost.localdom:8000 FIN_WAIT2
tcp 0 0 localhost.localdo:43711 localhost.localdom:8000 FIN_WAIT2
tcp 0 0 localhost.localdo:43717 localhost.localdom:8000 FIN_WAIT2
tcp 0 0 localhost.localdo:43777 localhost.localdom:8000 FIN_WAIT2
tcp 0 0 localhost.localdo:43722 localhost.localdom:8000 FIN_WAIT2
tcp 0 0 localhost.localdo:43725 localhost.localdom:8000 FIN_WAIT2
tcp6 0 0 [::]:8000 [::]:* LISTEN
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43724 CLOSE_WAIT
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43786 CLOSE_WAIT
tcp6 1 0 localhost.localdom:8000 localhost.localdo:43679 CLOSE_WAIT
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43735 CLOSE_WAIT
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43757 CLOSE_WAIT
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43754 CLOSE_WAIT
tcp6 83 0 localhost.localdom:8000 localhost.localdo:43723 CLOSE_WAIT
....
Since that no more requests are served by the server. One more detail: The same ab script with the same parameters works smoothly testing a simple Node.js server on the same machine. So this issue is not related to a number of opened TCP connections which I have set to be reusable with
sudo sysctl -w net.ipv4.tcp_tw_recycle=1
sudo sysctl -w net.ipv4.tcp_tw_reuse=1
Could anyone give me a clue on what I'm missing?
Edit: Termination of stream handling has been added to the code above:
if (line == null) {
println("Stream terminated")
return request
}
I'm posting the (partial) answer to my own question for those who will stumble upon the same issue one day. First, the nature of the problem lies not in the source code but in the system itself which restricts numerious connections. The problem is that the socket passed to readSocket function appears corrupted under some conditions, i.e. it can not be read and bufferedReader.readLine() either returns null on first call or hangs indefinitely. The following two steps make the code working on some machines:
Increase the number of concurrent connections to a socket with
sysctl -w net.core.somaxconn=65535
Provide the second parameter to ServerSocket constructor which will explicitly set the length of connection queue:
val maxQueue = 50000
val serverSocket = new ServerSocket(port, maxQueue)
The steps above solve the problem on EC2 m1.large instances, however I'm still getting issues on my local machine. The better way would be to use Akka for the stuff of that kind:
import akka.actor._
import java.net.InetSocketAddress
import akka.util.ByteString
class TCPServer(port: Int) extends Actor {
override def preStart {
IOManager(context.system).listen(new InetSocketAddress(port))
}
def receive = {
case IO.NewClient(server) =>
server.accept()
case IO.Read(rHandle, bytes) => {
val byteString = ByteString("HTTP/1.1 200 OK\r\n\r\nOK")
rHandle.asSocket.write(byteString)
rHandle.close()
}
}
}
object Application {
def main(args: Array[String]) {
val port = 8000
ActorSystem().actorOf(Props(new TCPServer(port)))
}
}
First, I'd suggest trying this without ab. You can do something like:
echo "I'm\nHappy\n" | nc -vv localhost 8000
Second, I'd suggest handling end-of-stream. This is where BufferedReader.readLine() returns null. The code above only checks for an empty String. After you fix this, I'd try again. Then test with ab, after everything looks good. Let us know if the problem persists.