Develop Reference

What are the major issues while using websocket to get stream of data?

I'm doing it first time. Where am going to read stream of data using websocket.
Here is my code snippet
RsvpApplication
#SpringBootApplication
public class RsvpApplication {
private static final String MEETUP_RSVPS_ENDPOINT = "ws://stream.myapi.com/2/rsvps";
public static void main(String[] args) {
SpringApplication.run(RsvpApplication.class, args);
}
#Bean
public ApplicationRunner initializeConnection(
RsvpsWebSocketHandler rsvpsWebSocketHandler) {
return args -> {
System.out.println("initializeConnection");
WebSocketClient rsvpsSocketClient = new StandardWebSocketClient();
rsvpsSocketClient.doHandshake(
rsvpsWebSocketHandler, MEETUP_RSVPS_ENDPOINT);
};
}
}
RsvpsWebSocketHandler
#Component
class RsvpsWebSocketHandler extends AbstractWebSocketHandler {
private static final Logger logger =
Logger.getLogger(RsvpsWebSocketHandler.class.getName());
private final RsvpsKafkaProducer rsvpsKafkaProducer;
public RsvpsWebSocketHandler(RsvpsKafkaProducer rsvpsKafkaProducer) {
this.rsvpsKafkaProducer = rsvpsKafkaProducer;
}
#Override
public void handleMessage(WebSocketSession session,
WebSocketMessage<?> message) {
logger.log(Level.INFO, "New RSVP:\n {0}", message.getPayload());
System.out.println("handleMessage");
rsvpsKafkaProducer.sendRsvpMessage(message);
}
}
RsvpsKafkaProducer
#Component
#EnableBinding(Source.class)
public class RsvpsKafkaProducer {
private static final int SENDING_MESSAGE_TIMEOUT_MS = 10000;
private final Source source;
public RsvpsKafkaProducer(Source source) {
this.source = source;
}
public void sendRsvpMessage(WebSocketMessage<?> message) {
System.out.println("sendRsvpMessage");
source.output()
.send(MessageBuilder.withPayload(message.getPayload())
.build(),
SENDING_MESSAGE_TIMEOUT_MS);
}
}
As far I know and read about websocket is that, It needs one time connection and stream of data will be flowing continuously until either party (client or server) stops.
I'm building it first time, so trying to cover major scenarios which can come acroos while dealing with 10000+ messages per minute. Total kafka brokers are two with enough space.
What can be done, if connection gets lost and again start consuming messages from webscoket once connected back where it was left in last failure and push messages into further Kafka broker ?
What can be done to put on hold websocket to keep pushing messages in broker if it has reached to threshold limit of not processed messages (in broker) ?
What can be done, When broker reached to its threshold, run a separate process to check available space in broker to push more messages and give indication to resume pushing messages in kafka broker ?
Please share other issues, which needs to be considered while setting up this thing ?

Java: Is there any way to run a method of a class that initialized the instance of the class from which I want to run it?

I'm using the JMS (Java Messaging System) to test a standard application between a Client and a Server.
The application is divided by layers, in the client module, I have 3 different files, there is the Main which is like the application itself, the file of the controller that has a ListView with the list of messages and finally the third file that basically is the one that is responsible for sending and receiving incoming messages.
As it is divided into layers, when I start the application, the Controller starts an instance of the Messaging class, so that it can send the messages that are written in the interface and of course store them in the ListView that I have with all message history.
The question is the following, when I send a message, it passes it from the controller to the class of the messaging so it could be sent to the server, but when I do the reverse process, that the messaging receives a new message, I can not find the way to pass it back to the controller, since the controller was the one who initiated the messaging instance and not the other way around.
THE CONTROLLER LOOKS THIS WAY:
public class LoanClientController implements Initializable{
public ApplicationGateway applicationGateway = new ApplicationGateway(){};
public TextField tfSsn;
public TextField tfAmount;
public TextField tfTime;
public ListView<ListViewLine> lvLoanRequestReply;
#Override
public void initialize(URL location, ResourceBundle resources) {
tfSsn.setText("123456");
tfAmount.setText("80000");
tfTime.setText("30");
}
#FXML
public void btnSendLoanRequestClicked() throws Exception {
int ssn = Integer.parseInt(tfSsn.getText());
int amount = Integer.parseInt(tfAmount.getText());
int time = Integer.parseInt(tfTime.getText());
LoanRequest loanRequest = new LoanRequest(ssn,amount,time);
ListViewLine listViewLine = applicationGateway.sendMessage(loanRequest);
lvLoanRequestReply.getItems().add(listViewLine);
applicationGateway.setLvLoanRequestReply(lvLoanRequestReply);
lvLoanRequestReply.refresh();
}
}
ON THE OTHER SIDE, THE APPLICATION GATEWAY a.k.a. "MESSAGING CLASS" is the following:
public class ApplicationGateway implements MessageListener {
public MessagingGateway messagingGateway = new MessagingGateway("Client-Broker", "Broker-Client");
public ApplicationGateway(){
try {
messagingGateway.getConsumer().setMessageListener(this);
} catch (JMSException e) {
e.printStackTrace();
}
}
#Override
public void onMessage(Message message) {
try {
Gson gson = new Gson();
LoanReply loanReply = gson.fromJson(((TextMessage) message).getText(), LoanReply.class);
} catch (JMSException e) {
e.printStackTrace();
}
}
}
Basically, what I'm trying to do is when a new message arrives via the onMessage method, send the value to the Controller.

Is there a way to stream a log to a Spring Boot application?

I have a Spring Boot app that is used as an event logger. Each client sends different events via a REST api, which are then saved in a database. But apart from simple events, I need the clients to also send their execution logs to Spring Boot.
Now, uploading a log after a client finishes executing is easy, and there are plenty examples for it out there. What I need is to stream the log as the client is executing, line by line, and not wait until the client has finished.
I've spent quite some time googling for a possible answer and I couldn't find anything that fits my needs. Any advice how to do this using Spring Boot (future releases included)? Is it feasible?

I see a couple of possibilities here. First, consider using a logback (the default Spring Boot logging implementation) SocketAppender or ServerSocketAppender in your client. See: https://logback.qos.ch/manual/appenders.html. This would let you send log messages to any logging service.
But I might suggest that you not log to your Spring Boot Event App as I suspect that will add complexity to your app unnecessarily, and I can see a situation where there is some bug in the Event App that then causes clients to log a bunch of errors which in turn all go back to the event app making it difficult to determine the initial error.
What I would respectfully suggest is that you instead log to a logging server - logstash: https://www.elastic.co/products/logstash for example, or if you already have a db that you are saving the event to, then maybe use the logbook DBAppender and write the logs directly to a db.

I wrote here an example on how to stream file updates in a spring boot endpoint. The only difference is that the code uses the Java WatchService API to trigger file updates on a given file.
However, in your situation, I would also choose the log appender to directly send messages to the connected clients (with sse - call template.broadcast from there) instead of watching for changes like I described.
The endpoint:
#GetMapping(path = "/logs", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public SseEmitter streamSseMvc() {
return sseService.newSseEmitter();
}
The service:
public class LogsSseService {
private static final Logger log = LoggerFactory.getLogger(LogsSseService.class);
private static final String TOPIC = "logs";
private final SseTemplate template;
private static final AtomicLong COUNTER = new AtomicLong(0);
public LogsSseService(SseTemplate template, MonitoringFileService monitoringFileService) {
this.template = template;
monitoringFileService.listen(file -> {
try {
Files.lines(file)
.skip(COUNTER.get())
.forEach(line ->
template.broadcast(TOPIC, SseEmitter.event()
.id(String.valueOf(COUNTER.incrementAndGet()))
.data(line)));
} catch (IOException e) {
e.printStackTrace();
}
});
}
public SseEmitter newSseEmitter() {
return template.newSseEmitter(TOPIC);
}
}
The custom appender (which you have to add to your logger - check here):
public class StreamAppender extends UnsynchronizedAppenderBase<ILoggingEvent> implements SmartLifecycle {
public static final String TOPIC = "logs";
private final SseTemplate template;
public StreamAppender(SseTemplate template) {
this.template = template;
}
#Override
protected void append(ILoggingEvent event) {
template.broadcast(TOPIC, SseEmitter.event()
.id(event.getThreadName())
.name("log")
.data(event.getFormattedMessage()));
}
#Override
public boolean isRunning() {
return isStarted();
}
}

Updating Dropwizard config at runtime

Is it possible to have my app update the config settings at runtime? I can easily expose the settings I want in my UI but is there a way to allow the user to update settings and make them permanent ie save them to the config.yaml file? The only way I can see it to update the file by hand then restart the server which seems a bit limiting.

Yes. It is possible to reload the service classes at runtime.
Dropwizard by itself does not have the way to reload the app, but jersey has.
Jersey uses a container object internally to maintain the running application. Dropwizard uses the ServletContainer class of Jersey to run the application.
How to reload the app without restarting it -
Get a handle to the container used internally by jersey
You can do this by registering a AbstractContainerLifeCycleListener in Dropwizard Environment before starting the app. and implement its onStartup method as below -
In your main method where you start the app -
//getting the container instance
environment.jersey().register(new AbstractContainerLifecycleListener() {
#Override
public void onStartup(Container container) {
//initializing container - which will be used to reload the app
_container = container;
}
});
Add a method to your app to reload the app. It will take in the list of string which are the names of the service classes you want to reload. This method will call the reload method of the container with the new custom DropWizardConfiguration instance.
In your Application class
public static synchronized void reloadApp(List<String> reloadClasses) {
DropwizardResourceConfig dropwizardResourceConfig = new DropwizardResourceConfig();
for (String className : reloadClasses) {
try {
Class<?> serviceClass = Class.forName(className);
dropwizardResourceConfig.registerClasses(serviceClass);
System.out.printf(" + loaded class %s.\n", className);
} catch (ClassNotFoundException ex) {
System.out.printf(" ! class %s not found.\n", className);
}
}
_container.reload(dropwizardResourceConfig);
}
For more details see the example documentation of jersey - jersey example for reload
Consider going through the code and documentation of following files in Dropwizard/Jersey for a better understanding -
Container.java
ContainerLifeCycleListener.java
ServletContainer.java
AbstractContainerLifeCycleListener.java
DropWizardResourceConfig.java
ResourceConfig.java

No.
Yaml file is parsed at startup and given to the application as Configuration object once and for all. I believe you can change the file after that but it wouldn't affect your application until you restart it.
Possible follow up question: Can one restart the service programmatically?
AFAIK, no. I've researched and read the code somewhat for that but couldn't find a way to do that yet. If there is, I'd love to hear that :).

I made a task that reloads the main yaml file (it would be useful if something in the file changes). However, it is not reloading the environment. After researching this, Dropwizard uses a lot of final variables and it's quite hard to reload these on the go, without restarting the app.
class ReloadYAMLTask extends Task {
private String yamlFileName;
ReloadYAMLTask(String yamlFileName) {
super("reloadYaml");
this.yamlFileName = yamlFileName;
}
#Override
public void execute(ImmutableMultimap<String, String> parameters, PrintWriter output) throws Exception {
if (yamlFileName != null) {
ConfigurationFactoryFactory configurationFactoryFactory = new DefaultConfigurationFactoryFactory<ReportingServiceConfiguration>();
ValidatorFactory validatorFactory = Validation.buildDefaultValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = Jackson.newObjectMapper();
final ConfigurationFactory<ServiceConfiguration> configurationFactory = configurationFactoryFactory.create(ServiceConfiguration.class, validator, objectMapper, "dw");
File confFile = new File(yamlFileName);
configurationFactory.build(new File(confFile.toURI()));
}
}
}

You can change the configuration in the YAML and read it while your application is running. This will not however restart the server or change any server configurations. You will be able to read any changed custom configurations and use them. For example, you can change the logging level at runtime or reload other custom settings.
My solution -
Define a custom server command. You should use this command to start your application instead of the "server" command.
ArgsServerCommand.java
public class ArgsServerCommand<WC extends WebConfiguration> extends EnvironmentCommand<WC> {
private static final Logger LOGGER = LoggerFactory.getLogger(ArgsServerCommand.class);
private final Class<WC> configurationClass;
private Namespace _namespace;
public static String COMMAND_NAME = "args-server";
public ArgsServerCommand(Application<WC> application) {
super(application, "args-server", "Runs the Dropwizard application as an HTTP server specific to my settings");
this.configurationClass = application.getConfigurationClass();
}
/*
* Since we don't subclass ServerCommand, we need a concrete reference to the configuration
* class.
*/
#Override
protected Class<WC> getConfigurationClass() {
return configurationClass;
}
public Namespace getNamespace() {
return _namespace;
}
#Override
protected void run(Environment environment, Namespace namespace, WC configuration) throws Exception {
_namespace = namespace;
final Server server = configuration.getServerFactory().build(environment);
try {
server.addLifeCycleListener(new LifeCycleListener());
cleanupAsynchronously();
server.start();
} catch (Exception e) {
LOGGER.error("Unable to start server, shutting down", e);
server.stop();
cleanup();
throw e;
}
}
private class LifeCycleListener extends AbstractLifeCycle.AbstractLifeCycleListener {
#Override
public void lifeCycleStopped(LifeCycle event) {
cleanup();
}
}
}
Method to reload in your Application -
_ymlFilePath = null; //class variable
public static boolean reloadConfiguration() throws IOException, ConfigurationException {
boolean reloaded = false;
if (_ymlFilePath == null) {
List<Command> commands = _configurationBootstrap.getCommands();
for (Command command : commands) {
String commandName = command.getName();
if (commandName.equals(ArgsServerCommand.COMMAND_NAME)) {
Namespace namespace = ((ArgsServerCommand) command).getNamespace();
if (namespace != null) {
_ymlFilePath = namespace.getString("file");
}
}
}
}
ConfigurationFactoryFactory configurationFactoryFactory = _configurationBootstrap.getConfigurationFactoryFactory();
ValidatorFactory validatorFactory = _configurationBootstrap.getValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = _configurationBootstrap.getObjectMapper();
ConfigurationSourceProvider provider = _configurationBootstrap.getConfigurationSourceProvider();
final ConfigurationFactory<CustomWebConfiguration> configurationFactory = configurationFactoryFactory.create(CustomWebConfiguration.class, validator, objectMapper, "dw");
if (_ymlFilePath != null) {
// Refresh logging level.
CustomWebConfiguration webConfiguration = configurationFactory.build(provider, _ymlFilePath);
LoggingFactory loggingFactory = webConfiguration.getLoggingFactory();
loggingFactory.configure(_configurationBootstrap.getMetricRegistry(), _configurationBootstrap.getApplication().getName());
// Get my defined custom settings
CustomSettings customSettings = webConfiguration.getCustomSettings();
reloaded = true;
}
return reloaded;
}

Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you.
Before I get started, note that this isn't a complete solution for the question asked as it doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.

How to get an existing websocket instance

I'm working on an application that uses Websockets (Java EE 7) to send messages to all the connected clients asynchronously. The server (Websocket endpoint) should send these messages whenever a new article (an engagement modal in my app) is created.
Everytime a connection is established to the websocket endpoint, I'm adding the corresponding session to a list, which I could be able to access outside.
But the problem I had is, when I'm accessing this created websocket endpoint to which all the clients connected from outside (any other business class), I've get the existing instance (like a singleton).
So, can you please suggest me a way I can get an existing instance of the websocket endpoint, as I can't create it as new MyWebsocketEndPoint() coz it'll be created by the websocket internal mechanism whenever the request from a client is received.
For a ref:
private static WebSocketEndPoint INSTANCE = null;
public static WebSocketEndPoint getInstance() {
if(INSTANCE == null) {
// Instead of creating a new instance, I need an existing one
INSTANCE = new WebSocketEndPoint ();
}
return INSTANCE;
}
Thanks in advance.

The container creates a separate instance of the endpoint for every client connection, so you can't do what you're trying to do. But I think what you're trying to do is send a message to all the active client connections when an event occurs, which is fairly straightforward.
The javax.websocket.Session class has the getBasicRemote method to retrieve a RemoteEndpoint.Basic instance that represents the endpoint associated with that session.
You can retrieve all the open sessions by calling Session.getOpenSessions(), then iterate through them. The loop will send each client connection a message. Here's a simple example:
#ServerEndpoint("/myendpoint")
public class MyEndpoint {
#OnMessage
public void onMessage(Session session, String message) {
try {
for (Session s : session.getOpenSessions()) {
if (s.isOpen()) {
s.getBasicRemote().sendText(message);
}
} catch (IOException ex) { ... }
}
}
But in your case, you probably want to use CDI events to trigger the update to all the clients. In that case, you'd create a CDI event that a method in your Websocket endpoint class observes:
#ServerEndpoint("/myendpoint")
public class MyEndpoint {
// EJB that fires an event when a new article appears
#EJB
ArticleBean articleBean;
// a collection containing all the sessions
private static final Set<Session> sessions =
Collections.synchronizedSet(new HashSet<Session>());
#OnOpen
public void onOpen(final Session session) {
// add the new session to the set
sessions.add(session);
...
}
#OnClose
public void onClose(final Session session) {
// remove the session from the set
sessions.remove(session);
}
public void broadcastArticle(#Observes #NewArticleEvent ArticleEvent articleEvent) {
synchronized(sessions) {
for (Session s : sessions) {
if (s.isOpen()) {
try {
// send the article summary to all the connected clients
s.getBasicRemote().sendText("New article up:" + articleEvent.getArticle().getSummary());
} catch (IOException ex) { ... }
}
}
}
}
}
The EJB in the above example would do something like:
...
#Inject
Event<ArticleEvent> newArticleEvent;
public void publishArticle(Article article) {
...
newArticleEvent.fire(new ArticleEvent(article));
...
}
See the Java EE 7 Tutorial chapters on WebSockets and CDI Events.
Edit: Modified the #Observer method to use an event as a parameter.
Edit 2: wrapped the loop in broadcastArticle in synchronized, per #gcvt.
Edit 3: Updated links to Java EE 7 Tutorial. Nice job, Oracle. Sheesh.

Actually, WebSocket API provides a way how you can control endpoint instantiation. See https://tyrus.java.net/apidocs/1.2.1/javax/websocket/server/ServerEndpointConfig.Configurator.html
simple sample (taken from Tyrus - WebSocket RI test):
public static class MyServerConfigurator extends ServerEndpointConfig.Configurator {
public static final MyEndpointAnnotated testEndpoint1 = new MyEndpointAnnotated();
public static final MyEndpointProgrammatic testEndpoint2 = new MyEndpointProgrammatic();
#Override
public <T> T getEndpointInstance(Class<T> endpointClass) throws InstantiationException {
if (endpointClass.equals(MyEndpointAnnotated.class)) {
return (T) testEndpoint1;
} else if (endpointClass.equals(MyEndpointProgrammatic.class)) {
return (T) testEndpoint2;
}
throw new InstantiationException();
}
}
You need to register this to an endpoint:
#ServerEndpoint(value = "/echoAnnotated", configurator = MyServerConfigurator.class)
public static class MyEndpointAnnotated {
#OnMessage
public String onMessage(String message) {
assertEquals(MyServerConfigurator.testEndpoint1, this);
return message;
}
}
or you can use it with programmatic endpoints as well:
public static class MyApplication implements ServerApplicationConfig {
#Override
public Set<ServerEndpointConfig> getEndpointConfigs(Set<Class<? extends Endpoint>> endpointClasses) {
return new HashSet<ServerEndpointConfig>
(Arrays.asList(ServerEndpointConfig.Builder
.create(MyEndpointProgrammatic.class, "/echoProgrammatic")
.configurator(new MyServerConfigurator())
.build()));
}
#Override
public Set<Class<?>> getAnnotatedEndpointClasses(Set<Class<?>> scanned) {
return new HashSet<Class<?>>(Arrays.asList(MyEndpointAnnotated.class));
}
Of course it is up to you if you will have one configurator used for all endpoints (ugly ifs as in presented snippet) or if you'll create separate configurator for each endpoint.
Please do not copy presented code as it is - this is only part of Tyrus tests and it does violate some of the basic OOM paradigms.
See https://github.com/tyrus-project/tyrus/blob/1.2.1/tests/e2e/src/test/java/org/glassfish/tyrus/test/e2e/GetEndpointInstanceTest.java for complete test.