Good evening. I got this little problem here.Im trying to connect two clients to a server. I made two queues where i put client1 and client2. I got this method here to read from the queue. But im only able to read from one of the queue.
NimMessage receiveMessage(Clientconnection client) throws NimServerException {
NimMessage request = null;
while (request == null){
request = (NimMessage) client1.toserver.pollLast(); //read from queue
}
//log("\n" + request.toString());
return request;
}
I enter the methodd with this
NimMessage request = receiveMessage(client1);
But when i want the second client to read the second queue with
request = receiveMessage(client2);
The receiveMessage method just reads from the client1 queue.
I cant figure out how to add the second queue in the receiveMessage method.
Related
I need to send my messages to Dead letter queue from azure topic subscription incase of any error while reading and processing the message from topic. So I tried testing pushing message directly to DLQ.
My sample code will be like
static void sendMessage()
{
// create a Service Bus Sender client for the queue
ServiceBusSenderClient senderClient = new ServiceBusClientBuilder()
.connectionString(connectionString)
.sender()
.topicName(topicName)
.buildClient();
// send one message to the topic
senderClient.sendMessage(new ServiceBusMessage("Hello, World!"));
}
static void resceiveAsync() {
ServiceBusReceiverAsyncClient receiver = new ServiceBusClientBuilder()
.connectionString(connectionString)
.receiver()
.topicName(topicName)
.subscriptionName(subName)
.buildAsyncClient();
// receive() operation continuously fetches messages until the subscription is disposed.
// The stream is infinite, and completes when the subscription or receiver is closed.
Disposable subscription = receiver.receiveMessages().subscribe(message -> {
System.out.printf("Id: %s%n", message.getMessageId());
System.out.printf("Contents: %s%n", message.getBody().toString());
}, error -> {
System.err.println("Error occurred while receiving messages: " + error);
}, () -> {
System.out.println("Finished receiving messages.");
});
// Continue application processing. When you are finished receiving messages, dispose of the subscription.
subscription.dispose();
// When you are done using the receiver, dispose of it.
receiver.close();
}
I tried getting the deadletter queue path
String dlq = EntityNameHelper.formatDeadLetterPath(topicName);
I got path of dead letter queue like = "mytopic/$deadletterqueue"
But It's not working while passing path as topic name. It throwing a Entity topic not found exception.
Any one can you please advise me on this
Reference :
How to move error message to Azure dead letter queue using Java?
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues#moving-messages-to-the-dlq
How to push the failure messages to Azure service bus Dead Letter Queue in Spring Boot Java?
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-java-how-to-use-topics-subscriptions-legacy#receive-messages-from-a-subscription
You probably know that a message will be automatically moved to the deadletter queue if you throw exceptions during processing, and the maximum delievery count is exceeded. If you want to explicitly move the message to the DLQ, you can do so as well. A common case for this is if you know that the message can never succeed because of its contents.
You cannot send new messages directly to the DLQ, because then you would have two messages in the system. You need to call a special operation on the parent entity. Also, <topic path>/$deadletterqueue does not work, because this would be the DLQ of all subscriptions. The correct entity path is built like this:
<queue path>/$deadletterqueue
<topic path>/Subscriptions/<subscription path>/$deadletterqueue
https://github.com/Azure/azure-service-bus/blob/master/samples/Java/azure-servicebus/DeadletterQueue/src/main/java/com/microsoft/azure/servicebus/samples/deadletterqueue/DeadletterQueue.java
This sample code is for queues, but you should be able to adapt it to topics quite easily:
// register the RegisterMessageHandler callback
receiver.registerMessageHandler(
new IMessageHandler() {
// callback invoked when the message handler loop has obtained a message
public CompletableFuture<Void> onMessageAsync(IMessage message) {
// receives message is passed to callback
if (message.getLabel() != null &&
message.getContentType() != null &&
message.getLabel().contentEquals("Scientist") &&
message.getContentType().contentEquals("application/json")) {
// ...
} else {
return receiver.deadLetterAsync(message.getLockToken());
}
return receiver.completeAsync(message.getLockToken());
}
// callback invoked when the message handler has an exception to report
public void notifyException(Throwable throwable, ExceptionPhase exceptionPhase) {
System.out.printf(exceptionPhase + "-" + throwable.getMessage());
}
},
// 1 concurrent call, messages are auto-completed, auto-renew duration
new MessageHandlerOptions(1, false, Duration.ofMinutes(1)),
executorService);
I'm trying to implement a simple Websocket application in Java that is able to scale horizontally, by using Redis and the Redisson library.
The Websocket server basically keeps track of connected clients, and sends message that are received to an Rtopic - this works great.
To consume, I have code that adds a listener when a client is registered : it associated a Client object with a listener by:
private static RedissonClient redisson = RedissonRedisServer.createRedisConnectionWithConfig();
public static final RTopic subcriberTopic = redisson.getTopic("clientsMapTopic");
public static boolean sendToPubSub(ConnectedClient q, String message) {
boolean[] success = {true};
MessageListener<Message> listener = new MessageListener<Message>() {
#Override
public void onMessage(CharSequence channel, Message message) {
logger.debug("The message is : " + message.getMediaId());
try {
logger.debug("ConnectedClient mediaid: " + q.getMediaid() + ",Message mediaid " + message.getMediaId());
if (q.getMediaid().equals(message.getMediaId())) {
// we need to verify if the message goes to the right receiver
logger.debug("MESSAGE from PUBSUB to (" + q.getId() + ") # " + q.getSession().getId() + " " + message);
// this is the actual message to the websocket client
// this executes on the wrong connected client when the connection is closed and reopened
q.getSession().getBasicRemote().sendText(message.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
success[0] = false;
}
}
};
int listenerId = subcriberTopic.addListener(Message.class, listener);
}
The problem I am observing is as follows:
initial connection from client registers listener associated with that object
sent message to the ws server gets picked up by listener and sent properly
disconnect websocket - create new connection - new listener gets created
sent message to the ws server gets picked up by same original listener and uses that connected client instead of the newly registered one
sending fails (because client and ws connection don't exist)and is not processes further
It seems I just need to remove the listener for the client if the client gets removed, but I haven't found a good way to do that because although I see in the debugger that the listener has the associated connected client object, I'm unable to retrieve them without adding code for that.
Am I observing this correctly and what is a good way to make this work properly?
When I was writing the question, I kind of leaned to an answer that I had in mind and tried, which worked.
I added a ConcurrentHashmap to keep track of the relation between the connected client and the listener.
In the logic where I handled websocket error that pointed to a client removing, I then removed the listener that was associated (and the entry from the map).
Now it works as expected.
small snippet:
int listenerId = subcriberTopic.addListener(Message.class, listener);
clientListeners.put(q,(Integer)listenerId);
And then in the websocket onError handler that triggers the cleanup:
// remove the associated listener
int listenerIdForClient = MessageContainer.clientListeners.get(cP);
MessageContainer.subcriberTopic.removeListener((Integer) listenerIdForClient);
// remove entry from map
MessageContainer.clientListeners.remove(cP);
Now the listener gets cleaned up properly and the next time a new listener gets created and handles the messages.
I am getting started with vertx and was trying out point to point messaging on event bus. I have 2 services both created as separate maven projects and deployed as fat-jars
1) Read from a file and send the content as a message over an address - ContentParserService.java
2) Read the message and reply to the incoming message- PingService.java
Both these services are deployed as separate jars kind of a microservice fashion
The code is as follows: ContentParserService.java
#Override
public void start(Future<Void> startFuture) throws Exception {
super.start(startFuture);
// Reference to the eventbus running on JVM
EventBus eventBus = vertx.eventBus();
// Read file using normal java mechanism
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(ClassLoader.getSystemResourceAsStream(
(config().getString("filename")))));
bufferedReader.readLine(); //read first line
String line = null;
while ((line = bufferedReader.readLine()) != null) {
String[] data = line.split(",");
// Create RealEstate Object
RealEstateTransaction realEstateData = createTransactionObject(data);
// Construct Message JSON
JsonObject messageJSON = constructMessageJson(realEstateData);
// Send message to PING address over the Event Bus
eventBus.send("PING", Json.encode(messageJSON), reply -> {
if (reply.succeeded())
System.out.println("Received Reply: " + reply.result().body());
else {
System.out.println("No reply");
}
});
}
} catch (IOException e) {
startFuture.fail(e.getMessage());
}
The code is as follows: PingService.java
#Override
public void start(Future<Void> startFuture) throws Exception {
super.start(startFuture);
System.out.println("Referencing event bus");
// Reference to the event bus running on the JVM
EventBus eventBus = vertx.eventBus();
System.out.println("Creating HttpServer");
// Create HTTP Server to handle incoming requests
HttpServer httpServer = vertx.createHttpServer();
System.out.println("Creating Router");
// Create Router for routing to appropriate endpoint
Router router = Router.router(vertx);
System.out.println("Starting to consume message sent over event bus");
// Consume the incoming message over the address PING
eventBus.consumer("PING", event -> {
System.out.println("Received message: " + event.body());
event.reply("Received at PING address");
});
System.out.println("Receiver ready and receiving messages");
When i run both the services I run on the same machine with the java -jar command for each of the service. What i observed was when i deploy the first jar of ContentParserService, it immediately starts and sends messages over the event bus, but by the time i start the pingservice jar , it is not able to receive any message sent over the event bus because my pingService is a separate fatjar and a microservice in itself. The file that i am reading is a finite lenght csv file of around 200 entries. This case would work if i bundle both the services in a single fat jar.
How should i achieve the different fat jars services able to send message to each other in my case.
This case works when both verticles in the same jar only because there's no network delay. But your usecase for EventBus is incorrect, since it doesn't persist messages, hence cannot replay them. You should start sending messages only when the other side is ready to receive them.
You need to reverse the dependency. In your ContentParserService register for some "ready" event, then start your while loop only when you get it:
vertx.eventBus().consumer("ready", (message) -> {
while ((line = bufferedReader.readLine()) != null) {
...
}
});
Now, what will happen if ContentParserService is actually slower and misses the "ready" event? Use vertx.setPeriodic() for that. So you start your PingService, and periodically tell ContentParserService that you're ready to receive some messages.
Or, as an option, just don't use EventBus at all between you services, and switch to something with persistence, like RabbitMQ or Kafka.
Priority & Dependency:
Here I made I simple test. But the result seems not so good.
I tried to make 100 request in a for loop in the same connection(the request url is the same, I am wondering whether this part influence the results).
If the index is i, then my request stream_id is i while the dependent stream_id is 100+i. If our assumption is right, the request can never get response because there is no stream_id from 101 to 200.
But the results shows there is no difference for setting the dependency and not. I got the response data frame one by one without timeout or waiting.
And also some other related test, the start point is to let the stream which depends on other stream to be sent first and the stream dependent later. But the result is same.
I am still thinking the reason of the results. Can anyone help me? Many thanks.
Code here:
public void run() throws Exception
{
host = "google.com";
port = 443;
//client init
HTTP2Client client = new HTTP2Client();
SslContextFactory sslContextFactory = new SslContextFactory(true);
client.addBean(sslContextFactory);
client.start();
//connect init
FuturePromise<Session> sessionPromise = new FuturePromise<>();
client.connect(sslContextFactory, new InetSocketAddress(host, port), new ServerSessionListener.Adapter(), sessionPromise);
Session session = sessionPromise.get(10, TimeUnit.SECONDS);
//headers init
HttpFields requestFields = new HttpFields();
requestFields.put("User-Agent", client.getClass().getName() + "/" + Jetty.VERSION);
final Phaser phaser = new Phaser(2);
//multiple request in one connection
for(int i=0;i<100;i++)
{
MetaData.Request metaData = new MetaData.Request("GET", new HttpURI("https://" + host + ":" + port + "/"), HttpVersion.HTTP_2, requestFields);
PriorityFrame testPriorityFrame = new PriorityFrame(i, 100+i, 4, true);
HeadersFrame headersFrame = new HeadersFrame(0, metaData, testPriorityFrame, true);
//listen header/data/push frame
session.newStream(headersFrame, new Promise.Adapter<Stream>(), new Stream.Listener.Adapter()
{
#Override
public void onHeaders(Stream stream, HeadersFrame frame)
{
System.err.println(frame+"headId:"+frame.getStreamId());
if (frame.isEndStream())
phaser.arrive();
}
#Override
public void onData(Stream stream, DataFrame frame, Callback callback)
{
System.err.println(frame +"streamid:"+ frame.getStreamId());
callback.succeeded();
if (frame.isEndStream())
phaser.arrive();
}
#Override
public Stream.Listener onPush(Stream stream, PushPromiseFrame frame)
{
System.err.println(frame+"pushid:"+frame.getStreamId());
phaser.register();
return this;
}
});
}
phaser.awaitAdvanceInterruptibly(phaser.arrive(), 5, TimeUnit.SECONDS);
client.stop();
}
The Jetty project did not implement (yet) HTTP/2 request prioritization.
We are discussing whether this is any useful for a server, whose concern is to write back the responses as quick as it can.
Having one client changing its mind on the priority of the requests, or making a request knowing that in reality it first wanted another request served, it's a lot of work for the server that in the meantime has to serve the other 10,000 clients connected to it.
By the time we the server has recomputed the priority tree for the dependent requests, it could have probably have served the requests already.
By the time the client realizes that it has to change the priority of a request, the whole response for it could already be in flight.
Having said that, we are certainly interested in real world use cases where request prioritization performed by the server yields a real performance improvement. We just have not seen it yet.
I would love to hear why you are interested in request prioritization and how you are leveraging it. Your answer could be a drive for the Jetty project to implement HTTP/2 priorities.
I am writing a Java HTTP server. I thought the entire server was working and it is using threading. However, I'm realizing that the piece of code that reads the request into a BufferedReader is not working consistently.
Here is the code that reads an incoming request:
private String receive(WebSocket webSocket) throws IOException {
int chr;
System.out.println("Receiving!");
StringBuffer buffer = new StringBuffer();
while ( (chr = webSocket.in().read() ) != -1) {
buffer.append((char) chr);
if ( !webSocket.in().ready())
break;
}
return buffer.toString();
}
My Websocket class just wraps the Socket and provides an in and an out. I did this so that I could mock out the socket and test my server.
The Websocket class looks like this:
package http.server.socket;
import java.io.*;
import java.net.Socket;
public class SystemSocket implements WebSocket {
private Socket theConnection;
private BufferedReader in;
private OutputStream out;
public SystemSocket(Socket theConnection) throws IOException {
this.theConnection = theConnection;
in = new BufferedReader(new InputStreamReader(theConnection.getInputStream()));
out = new BufferedOutputStream(theConnection.getOutputStream());
}
public BufferedReader in() throws IOException {
return in;
}
public OutputStream out() throws IOException {
return out;
}
public void close() throws IOException {
in.close();
out.close();
theConnection.close();
}
}
The problem is that with each url the user enters in a browser, two requests are made - one for the page requested and one for the favicon. Sometimes - it seems - the favicon request is not coming in and the thread hangs.
Here's some debugging information I have printing to the console when things go right:
Receiving!
Receiving!
REQUEST STRING = GET /color_picker.html HT
[20130821 20:29:23] REQUEST: http://localhost:5000/color_picker.html
[20130821 20:29:23] PAGE RENDERED
REQUEST STRING = GET /favicon.ico HTTP/1.1
[20130821 20:29:23] REQUEST: http://localhost:5000/favicon.ico
[20130821 20:29:23] PAGE RENDERED
The "Receiving" message is getting printed whenever the request is getting read. So, in this case, the "Receiving" message got printed twice, two requests came in and two things were rendered. But then, the same page (but at a different time) will do this (after about 10 seconds):
Receiving!
Receiving!
REQUEST STRING = GET /color_picker.html HTTP/1.1
[20130821 20:41:25] REQUEST: http://localhost:5000/color_picker.html
[20130821 20:41:25] PAGE RENDERED
REQUEST STRING =
Exception in thread "ServerThread" java.lang.ArrayIndexOutOfBoundsException: 1
at http.request.Parser.setRequestLineData(Parser.java:42)
at http.request.Parser.setRequestHash(Parser.java:27)
at http.request.Parser.parse(Parser.java:13)
at http.request.Request.get(Request.java:18)
at http.server.ServerThread.run(ServerThread.java:39)
All the subsequent errors are because the request string is null. But I can't figure out why the Request string is null. I can't even figure out how to debug.
Can anyone help??
Also important to note that if the second request string doesn't come in right away, the user can request a new url and it will cause the second hung process to complete (so then the fourth request url will be what hangs). So, it's only when the user stops requesting things, on the last request after about 10 seconds, I will get the error. Sometimes I can request 20 different pages and it's only after I stop requesting pages and wait a few seconds, that I will see an error. I think this is what is happening??
UPDATE:
Per the request, here is the setRequestLineData() method:
private void setRequestLineData() {
requestHash = new HashMap<String, String>();
if (requestLineParts.length == 3) {
requestHash.put("httpMethod", requestLineParts[0]);
requestHash.put("url", requestLineParts[1]); //line 42
requestHash.put("httpProtocol", requestLineParts[2]);
}
else {
requestHash.put("httpMethod", requestLineParts[0]);
requestHash.put("url", requestLineParts[1]);
requestHash.put("queryString", requestLineParts[2]);
requestHash.put("httpProtocol", requestLineParts[3]);
}
}
UPDATE:
I think I figured out more about what is going on here with my mentor's help. His thought is that once a request is received, the browser starts another request right away to reduce load time for the next request. This sound plausible to me since I can load page after page after page, but it's only about 10 seconds after the last page is requested that I get an error. Currently, I'm handling this with a custom exception, but am working on a better solution. Thanks for all the help guys!
ready() isn't a valid test for end of message. It only tells you whether there is data available to be read without blocking. TCP isn't a message-oriented protocol, it is a byte-stream protocol. If you want messages you must implement them yourself, e.g. as lines, length-value tuples, type-length-value tuples, serialized objects, XML documents, ...
There are few if any correct uses of ready() (or available()), and this isn't one of them.