I have an application with websocket and stomp as the protocol for messaging. I followed the official spring documentation in order to create this application. After a couple of days, i got a report that the host in which the application is running received an alert from Prometheus called FdExhaustionClose, which from my understanding means that some connections are not being properly closed.
The application is running in kubernetes (linux) and we are using RabbitMQ as the message broker.
How can i fix this ? Running locally i realized that the total number of connections since the application started actually matches the number of file descriptors unclosed.
I checked the logs in production and the broker status was printing this:
WebSocketSession[42 current WS(42)-HttpStream(0)-HttpPoll(0), 52766 total, 0 closed abnormally
And a simply lsof -p PID | wc -l returned the number 52752
WebSocketConfig
//...
#Configuration
public class WebSocketChatConfig extends DelegatingWebSocketMessageBrokerConfiguration {
#Autowired
private ApplicationProperties properties;
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/wss").setAllowedOrigins("*");
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
RabbitMqProperties rabbitProperties = properties.getRabbitmq();
ReactorNettyTcpClient<byte[]> client = new ReactorNettyTcpClient<>(tcpClient -> tcpClient
.host(rabbitProperties.getHost())
.port(rabbitProperties.getPort())
.option(ChannelOption.SO_TIMEOUT, 3600000)
.noProxy()
.secure(SslProvider.defaultClientProvider()), new StompReactorNettyCodec());
registry.setApplicationDestinationPrefixes("/app", "/topic", "/chat");
registry.enableStompBrokerRelay("/topic")
.setRelayHost(rabbitProperties.getHost())
.setRelayPort(rabbitProperties.getPort())
.setVirtualHost(rabbitProperties.getUsername())
.setSystemLogin(rabbitProperties.getUsername())
.setSystemPasscode(rabbitProperties.getPassword())
.setClientLogin(rabbitProperties.getUsername())
.setClientPasscode(rabbitProperties.getPassword()).setTcpClient(client);
}
}```
After some research i found out that the problem was related to the package reactor-netty, apparently there was a leak in version 0.9.8, the leak got fixed in version 0.9.9
Related
I am using spring boot 2.7.7 and below are my maven dependencies:
Here, is how I set the DatadogConfig in my controller:
DatadogConfig config = new DatadogConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
public String get(String k) {
return null; // accept the rest of the defaults
}
};
logger.info(appConfig.getApiKey());
MeterRegistry registry = new DatadogMeterRegistry(config, Clock.SYSTEM);
My application.yml is set as below:
management.metrics.export.datadog:
apiKey: "******"
# You will probably want disable Datadog publishing in a local development profile.
enabled: true
# The interval at which metrics are sent to Datadog. The default is 1 minute.
step: "1m"
server:
port: 8081
After running the application, I do see that the Meter registry is trying to push metrics, however after sending a request to controller, I get the exception as shown below:
i.m.c.instrument.push.PushMeterRegistry : publishing metrics for DatadogMeterRegistry every 1m
i.m.datadog.DatadogMeterRegistry : An application key must be configured in order for unit information to be sent to Datadog.
........
io.micrometer.core.instrument.config.validate.ValidationException: datadog.apiKey was 'null' but it is required
at io.micrometer.core.instrument.config.validate.Validated$Either.orThrow(Validated.java:375) ~[micrometer-core-1.9.6.jar:1.9.6]
I did follow these posts, but still getting the error
Micrometer
SO
What config am I missing or doing wrong?
It's been days that I struggle to make a basic Client / server communication using websockets
My client is a java client with libGDX and my server is a basic java server
My main goal is to compile my client into HTML5 to communicate with my server using websockets.
I tryed the following solution when searching on google :
https://github.com/czyzby/gdx-lml/tree/master/websocket
https://github.com/pepedeab/libGDX-Net
https://github.com/TooTallNate/Java-WebSocket
The 1 seemed to be the best solution but, it doesn't seems to have the TCP_NODELAY socket setting (which is essential in my case)
The 2 seemed an other good solution too, But it relies on http://code.google.com/p/gwt-ws/(which at this time don't understand the point of this)
The 3 is what I choosed, a simple WebSocket Java API to let me write client and server really easily.
It worked very well for desktop and android, but when I tryed to html:dist, gradle give me error about websocket which was not inherit etc...
My main build.gradle file contains this line for each project (core, desktop, android, html) : compile "org.java-websocket:Java-WebSocket:1.3.7"
So to resume my primary question : How to correctly establish a websocket connection with a client compiled with GWT in ligdx, with in addition TCP_NODELAY?
My client is a very simple class :
package com.mygdx.game;
import java.net.URI;
import java.nio.ByteBuffer;
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.drafts.Draft;
import org.java_websocket.handshake.ServerHandshake;
public class WebSocketsNet extends WebSocketClient {
public WebSocketsNet(URI serverUri, Draft draft) {
super(serverUri, draft);
}
public WebSocketsNet(URI serverURI) {
super(serverURI);
}
#Override
public void onOpen(ServerHandshake handshakedata) {
send("Hello, it is me. Mario :)");
System.out.println("new connection opened");
}
#Override
public void onClose(int code, String reason, boolean remote) {
System.out.println("closed with exit code " + code + " additional info: " + reason);
}
#Override
public void onMessage(String message) {
System.out.println("received message: " + message);
}
#Override
public void onMessage(ByteBuffer message) {
System.out.println("received ByteBuffer");
}
#Override
public void onError(Exception ex) {
System.err.println("an error occurred:" + ex);
}}
According to https://bugs.webkit.org/show_bug.cgi?id=102079 and https://groups.google.com/forum/#!topic/native-client-discuss/T8zdrMjiTAE, found via https://github.com/websockets/ws/issues/791 and https://github.com/varspool/Wrench/pull/104, most browsers already use TCP_NODELAY. At least from the websocket standard, there is nothing you can do to influence this on the client - on the server there may be more options.
If TCP_NODELAY is already set on the client, you can set it on the server as well to ensure both sides are sending small messages as soon as possible.
Another thought that is suggested in questions like https://stackoverflow.com/a/13406438/860630 is to respond to every message sent right away, so that the network stack flushes all remaining messages as soon as possible.
Finaly I found a way to make it work, so I post a answer here for those interested.
I used https://github.com/czyzby/gdx-lml/tree/master/websocket
Particulary the example package, and rigorously follow everything that need to be added on build.gradle and on differents xml files, so now it work !
So to conclude :
Server Listening websockets with java-web-socket
LIBGDX client use gdx-websockets to connect to the server (Watch-out for different build gradle file and xml !!)
Hope to help some people who were in the same problem like me !
My smartphone collects GPS, Bluetooth log, then periodically send the data to server.
My server continuously receive the data by using Restlet.
However i encounter a error which i have never seen before and Google does not give any solution or hints. (my server has worked well for the past few days.)
Following message is errors i encountered.
Unable to run the following server-side task: sun-net.httpserver.ServerImpl$Exchange#81a5dc
Unable to run the following server-side task: sun-net.httpserver.ServerImpl$Exchange#~~~~~~
Unable to run the following server-side task: sun-net.httpserver.ServerImpl$Exchange#~~~~~~
Following is my code.
RestletServerMain.java
public void restServer(){
try{
Component component = new Component();
component.getServers().add(Protocol.HTTP, Integer.parseInt(Common.SERVER_PORT));
component.getDefaultHost().attach(new ServerApplication());
component.start();
}catch(Exception e){
e.printStackTrace();
}
}
ServerApplication.java
public class ServerApplication extends Application {
public Restlet createInboundRoot() {
Router router = new Router(getContext());
router.attach("/dataprocessing1", xxx.class);
router.attach("/dataprocessing2", yyy.class);
return router;
}
It could be interesting to try the jetty extension (org.restlet.extends.jetty) of Restlet (instead of the default one).
Just add the corresponding jar file in your classpath and Jetty will be used as underlying server for your application.
Hope it will fix your issue.
Thierry
I'm trying to implement some Websocket functionality for my web app running on Tomcat 7. I'm using the following tech:
(server) Spring Websocket + Spring Messaging
(client) SockJS + Stomp.js
I'm following this guide (roughly): http://g00glen00b.be/spring-angular-sockjs/
My configuration is similar to the guide, but I'll add some snippets of my code here:
Client:
s.socket = new SockJS('http://localhost:8181/.cckiosk/socket/test');
//s.socket = new SockJS('http://localhost:8181/.cckiosk/socket/test', {}, { transports: ['xhr-polling'] });
s.client = Stomp.over(s.socket);
s.client.connect({}, onConnect);
s.client.onclose = onDisconnect;
Server:
#Configuration
#ComponentScan
#EnableWebSocketMessageBroker
public class ModuleWebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureMessageBroker(MessageBrokerRegistry broker) {
//Prefix for messages FROM server TO client
broker.enableSimpleBroker("/client");
//Prefix for messages FROM client TO server
broker.setApplicationDestinationPrefixes("/server");
}
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/socket/test").setAllowedOrigins("*").withSockJS();
}
}
Controller:
#MessageMapping("/socket/test")
#SendTo("/client/message")
public GenericMessage doSample(GenericMessage msg) {
log.info("doSample: " + JsonUtil.jsonify(msg));
return new GenericMessage(msg.getId(), msg.getMessage(), new Date());
}
I have everything set up in a vanilla Spring app, and everything works great.
However, when I port the same code over to a Magnolia module, the code stops working and I can see the following errors:
Client-side error:
Server-side error:
2015-09-19 16:28:43,412 DEBUG eb.socket.handler.LoggingWebSocketHandlerDecorator: New WebSocketServerSockJsSession[id=tde1syjd]
2015-09-19 16:28:43,413 DEBUG eb.socket.handler.LoggingWebSocketHandlerDecorator: Transport error in WebSocketServerSockJsSession[id=tde1syjd]
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.coyote.http11.upgrade.BioServletInputStream.doRead(BioServletInputStream.java:37)
at org.apache.coyote.http11.upgrade.AbstractServletInputStream.read(AbstractServletInputStream.java:129)
at org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:47)
at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsReadListener.onDataAvailable(WsHttpUpgradeHandler.java:203)
at org.apache.coyote.http11.upgrade.AbstractServletInputStream.onDataAvailable(AbstractServletInputStream.java:203)
at org.apache.coyote.http11.upgrade.AbstractProcessor.upgradeDispatch(AbstractProcessor.java:93)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:623)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
It's actually either Connection Reset or Broken Pipe.
Changing the transport protocol to a non-websocket one (e.g. xhr-polling) also doesn't help.
Any idea what could be in Magnolia that's causing a long-lived Websocket session to have its connection closed?
The solution lay in simply bypassing the Magnolia filter chain altogether.
I just needed to create a global bypass object in Magnolia admincentral: Configuration > server/filters/bypasses
class: info.magnolia.StartsWithURIVoter
pattern: /socket
And the connection doesn't reset or get interrupted anymore.
In Mangolia 5 it is 'info.magnolia.voting.voters.URIStartsWithVoter', instead of 'info.magnolia.StartsWithURIVoter'
I am trying to get started with WebSockets, and trying to write a simple application to send messages back and forth via a websoket.
However, it looks like the socket that I am trying to create never gets connected. Why can that be?
Below is the code of my WebSockets class. When .onConnect() is called, it logs:
I am socket, I was connected. Am i connected? - false
Update: in JavaScript, where I create the socket in question, the readyState is 1, which means "socket open, communication is possble".
import a.b.Misc; //writes logs.
import com.sun.grizzly.websockets.BaseServerWebSocket;
import com.sun.grizzly.websockets.DataFrame;
import com.sun.grizzly.websockets.WebSocketListener;
public class ChatWebSocket_v2 extends BaseServerWebSocket {
private String user;
public ChatWebSocket_v2(WebSocketListener... listeners) {
super(listeners);
}
public String getUser() {
if (user == null) {
Misc.print("User is null in ChatWebSocket");
throw new NullPointerException("+=The user is null in chat web socket");
}
return user;
}
public void setUser(String user) {
Misc.print("Just set user: " + user);
this.user = user;
}
#Override
public void onMessage(String message) {
Misc.print(message +"\n");
}
#Override
public void onMessage(byte[] message) {
Misc.print(new String(message) +" << Bytes\n");
}
#Override
public void onConnect() {
Misc.print("I am socket, i was connected. Am i connected? - " + this.isConnected());
}
#Override
public void onClose(DataFrame df) {
Misc.print("I am socket, i was closed");
}
}
If you're just trying to make a connection somewhere, you might want to try this instead. There is a live working demo and you can download the javascript code and play with it yourself. Note that the javascript code only works if you have it installed on a server (due to browser security because it's 'fancy'.) There is also a step by step browser-based client tutorial in the works that I will post as soon as it's ready. Most proxy servers haven't been upgraded to handle websockets so they will screw up connection request and most people won't be able to connect to websocket servers from work. Firefox 7 (release) or Google Chrome 14 or later support the latest version of the websocket protocol that the demo server runs.
If you want to try to get the grizzly demo working, you might have some debugging to do and maybe I'll help with that. Note that in comments below the article, other people said they couldn't get it working either and I haven't found any follow up. At this point it seems no better than the echo app above even if we do get it running and is possibly overly complicated and underly documented if you're just trying to get started. But if you want to try to get it running, you should 'git' the latest version of the code here, which was at least committed recently and may be fixed.
Then make sure that app.url in the application javascript file is set to your installation directory. His is hard-coded as:
url: 'ws://localhost:8080/grizzly-websockets-chat/chat',
If you're using Firefox 7, the javascript needs to be modified to use the Moz prefix, for example:
if (typeof MozWebSocket != "undefined") { // window.MozWebSocket or "MozWebSocket" in window
ok
} else if (window.WebSocket) { // he uses if ("WebSocket" in window)
ok
} else {
do your print "browser doesn't support websockets"
}
.... then if the browser supports websockets
websocket = new WebSocket(app.url); or
websocket = new MozWebSocket(app.url);
// depending on which it is.
The HLL websocket server demo code has this all sorted out.
(another) UPDATE: As I work through grizzly myself, I found on the Quick Start in the glassfish admin console, there's a hello sample that's pretty easy to set up and run. You'll find instructions there. The sample directory also contains a war file named: websocket-mozilla; so I guess its supposed to use websockets. Someone who's familiar with jsp should review the source code. All I can see is that it's using an http session. No mention of a websocket at all. It's a lot like the hello sample.