Develop Reference

How to avoid Stack Overflow errors in infinitely recursive method?

I know there are tons of posts about stack overflow errors and i understand why my specific one is happening, my question is basically how to move away from recursion in this specific case. I have a class which establishes and maintains a client connection (for HL7 messaging specifically but it's essentially a glorified client connection) to another system which hosts corresponding server connections. This class' constructor starts a new thread and runs the following method :
#Override
public void connect()
{
try
{
setStatus("Connecting");
connection = context.newClient(intfc.getIp(), port, false);
connected = true;
setStatus("Connected");
logEntryService.logInfo(LogEntry.CONNECTIVITY, "Successfully connected " + connectionType + " client connection to "
+ intfc.getName() + "(" + intfc.getIp() + ") on port " + port);
monitor();
}
catch (HL7Exception ex)
{
connected = false;
setStatus("Disconnected");
try
{
TimeUnit.SECONDS.sleep(connectionRetryIntervalInSeconds);
connect();
}
catch (InterruptedException ex2)
{}
}
}
Upon successfully connecting with the server, the monitor method simply checks, in yet another thread, if the connection is still up at a given interval. If it goes down, the monitoring thread is interrupted and the connect() method is called again.
I did not anticipate this at first but you can quickly see why the connect() method is causing stack overflow errors after several days running. I'm struggling to think of a way to get the same functionality to work without the connect method calling itself again every time the connection fails.
Any suggestions would be appreciated.
Thanks!

Typically you'd use a Stack object to emulate recursion when required.
However, in your case, why are you using recursion at all? A while loop fits the purpose.
while(true /**or some relevant condition**/){
try{ //try to connect
....
catch(HL7Exception ex){
//sleep
}
}
I'm not sure of the purpose of your application, but there are may be better methods than sleeping. You could use a ScheduledExecutorService, but if it's a single threaded program with one purpose it's probably unnecessary.

When I had to deal with this issue in c# I used a Stack, and added new classes to it, instead of using recursion. Then a second loop would check to see if there were any objects in the stack that needed dealing with. That avoided stack overflow when I would have had huge amounts of recursion otherwise. Is there a similar Stack collection in Java?

Why are you calling the monitor() method in the first place? You mention that it is launched in a separate thread, then can't you just launch it in a new thread when the application comes up? Then there won't be a recursive call.

I changed my code to an iterative approach as suggested, works beautifully!
#Override
public void initThread()
{
initConnectionEntity();
mainThread = new Thread()
{
#Override
public void run()
{
while (running)
{
if (!connected)
{
try
{
connect();
}
catch (HL7Exception ex)
{
connected = false;
setStatus("Disconnected");
try
{
TimeUnit.SECONDS.sleep(connectionRetryIntervalInSeconds);
}
catch (InterruptedException ex2)
{}
}
}
try
{
TimeUnit.MILLISECONDS.sleep(500);
}
catch (InterruptedException ex2)
{}
}
}
};
mainThread.setName(intfc.getName() + " " + connectionType + " Main Thread");
mainThread.start();
}
#Override
public void connect() throws HL7Exception
{
setStatus("Connecting");
connection = context.newClient(intfc.getIp(), port, false);
connected = true;
setStatus("Connected");
logEntryService.logInfo(LogEntry.CONNECTIVITY, "Successfully connected " + connectionType + " client connection to "
+ intfc.getName() + "(" + intfc.getIp() + ") on port " + port);
monitor();
}
private void monitor()
{
monitoringThread = new Thread()
{
#Override
public void run()
{
try
{
while (running)
{
if (!connection.isOpen())
{
if (connected == true)
{
logEntryService.logWarning(LogEntry.CONNECTIVITY, "Lost " + connectionType + " connection to "
+ intfc.getName() + "(" + intfc.getIp() + ") on port " + port);
}
connected = false;
setStatus("Disconnected");
monitoringThread.interrupt();
}
else
{
connected = true;
}
TimeUnit.SECONDS.sleep(connectionMonitorIntervalInSeconds);
}
}
catch (InterruptedException ex)
{
logEntryService.logDebug(LogEntry.CONNECTIVITY, "Monitoring thread for " + connectionType
+ " connection to " + intfc.getName() + " interrupted");
}
}
};
monitoringThread.setName(intfc.getName() + " " + connectionType + " Monitoring Thread");
monitoringThread.start();
}

Why are threads using the same variable value? RxJava Mqtt

I'm using rxmqtt (which uses rxjava and paho) to communicate whith a mqtt broker. I'm using javax to accept rest requests and publish some content to the broker and wait for a response. The code below works fine if I make one request at a time, but if I have more than one concurrent requests it only returns a response for the last one and the others fall into the timeout exception.
The mqttConn.getMqttMessages() returns a flowable which is already subscribed to all topics i need:
public Flowable<MqttMessage> getMqttMessages() {
return this.obsClient.subscribe("pahoRx/fa/#", 1);
}
and MqttConnection is a singleton because i only want one single connection to broker and all the publishes are done in this connection
I've noticed that my queryParam id is different in each thread execution of the web service request (expected behavior), but when it enters the subscription part of the code it only considers the last id value and does not pass my validation in the takeUntil method:
mqttConn.getMqttMessages().timeout(20, TimeUnit.SECONDS).takeUntil(msgRcv -> {
System.out.println("received: " + new String(msgRcv.getPayload()) + " ID: " + id);
return id.equals(new String(msgRcv.getPayload()));
}).blockingSubscribe(msgRcv -> {
final byte[] body = msgRcv.getPayload();
System.out.println(new String(body)); //printing... but not sending the reponse
response.set("Message Receiced: " + new String(msgRcv.getPayload()));
return;
}, e -> {
if (e instanceof TimeoutException) {
response.set("Timeout Occured");
} else {
response.set("Some kind of error occured " + e.getLocalizedMessage());
}
});
The thing is, why is it only considering the last id received when each request should have its own independent thread? I've tried getting mqttConn.getMqttConnection() as a ThreadLocal object... doesn't fix.
Full WS code:
#Path("/test")
#GET
public String test(#QueryParam("id") String id) throws InterruptedException, MqttException {
String funcExec = "pahoRx/fe/";
String content = "unlock with single connection to broker";
int qos = 1;
AtomicReference<String> response = new AtomicReference<String>();
response.set("Initial Value");
MqttConnection mqttConn = MqttConnection.getMqttConnection();
ObservableMqttClient obsClient = mqttConn.getBrokerClient();
MqttMessage msg = MqttMessage.create(78, content.getBytes(), qos, false);
String topicPub = funcExec + id;
obsClient.publish(topicPub, msg).subscribe(t -> {
System.out.println("Message Published");
}, e -> {
System.out.println("Failed to publish message: " + e.getLocalizedMessage());
});
mqttConn.getMqttMessages().timeout(20, TimeUnit.SECONDS).takeUntil(msgRcv -> {
System.out.println("received: " + new String(msgRcv.getPayload()) + " ID: " + id);
return id.equals(new String(msgRcv.getPayload()));
}).blockingSubscribe(msgRcv -> {
final byte[] body = msgRcv.getPayload();
System.out.println(new String(body)); //printing... but not sending the reponse
response.set("Message Receiced: " + new String(msgRcv.getPayload()));
return;
}, e -> {
if (e instanceof TimeoutException) {
response.set("Timeout Occured");
} else {
response.set("Some kind of error occured " + e.getLocalizedMessage());
}
});
return response.get();
}
I hope the explanation is clear enough!
Ty in advance

Solr Trigger Optimize And Check Progress From Java Code

From this topic there are two ways to trigger solr optimize from Java code. Either sending an http request, or using solrj api.
But how to check the progress of it?
Say, an api which returns the progress of optimize in percentage
or strings like RUNNING/COMPLETED/FAILED.
Is there such an api?

Yes, optimize() in solrj api is a sync method. Here is what I used to monitor the optimization progress.
CloudSolrClient client = null;
try {
client = new CloudSolrClient(zkClientUrl);
client.setDefaultCollection(collectionName);
m_logger.info("Explicit optimize of collection " + collectionName);
long optimizeStart = System.currentTimeMillis();
UpdateResponse optimizeResponse = client.optimize();
for (Object object : optimizeResponse.getResponse()) {
m_logger.info("Solr optimizeResponse" + object.toString());
}
if (optimizeResponse != null) {
m_logger.info(String.format(
" Elapsed Time(in ms) - %d, QTime (in ms) - %d",
optimizeResponse.getElapsedTime(),
optimizeResponse.getQTime()));
}
m_logger.info(String.format(
"Time spent on Optimizing a collection %s :"
+ (System.currentTimeMillis() - optimizeStart)
/ 1000 + " seconds", collectionName));
} catch (Exception e) {
m_logger.error("Failed during explicit optimize on collection "
+ collectionName, e);
} finally {
if (client != null) {
try {
client.close();
} catch (IOException e) {
throw new RuntimeException(
"Failed to close CloudSolrClient connection.", e);
}
client = null;
}
}

concurrent modification on arraylist

There are a lot of concurrent mod exception questions, but I'm unable to find an answer that has helped me resolve my issue. If you find an answer that does, please supply a link instead of just down voting.
So I originally got a concurrent mod error when attempting to search through an arraylist and remove elements. For a while, I had it resolved by creating a second arraylist, adding the discovered elements to it, then using removeAll() outside the for loop. This seemed to work, but as I used the for loop to import data from multiple files I started getting concurrent modification exceptions again, but intermittently for some reason. Any help would be greatly appreciated.
Here's the specific method having the problem (as well as the other methods it calls...):
public static void removeData(ServiceRequest r) {
readData();
ArrayList<ServiceRequest> targets = new ArrayList<ServiceRequest>();
for (ServiceRequest s : serviceQueue) {
//ConcurrentModification Exception triggered on previous line
if (
s.getClient().getSms() == r.getClient().getSms() &&
s.getTech().getName().equals(r.getTech().getName()) &&
s.getDate().equals(r.getDate())) {
JOptionPane.showMessageDialog(null, s.getClient().getSms() + "'s Service Request with " + s.getTech().getName() + " on " + s.getDate().toString() + " has been removed!");
targets.add(s);
System.out.print("targetted"); }
}
if (targets.isEmpty()) { System.out.print("*"); }
else {
System.out.print("removed");
serviceQueue.removeAll(targets);
writeData(); }
}
public static void addData(ServiceRequest r) {
readData();
removeData(r);
if (r.getClient().getStatus().equals("MEMBER") || r.getClient().getStatus().equals("ALISTER")) {
serviceQueue.add(r); }
else if (r.getClient().getStatus().equals("BANNED") || r.getClient().getStatus().equals("UNKNOWN")) {
JOptionPane.showMessageDialog(null, "New Request failed: " + r.getClient().getSms() + " is " + r.getClient().getStatus() + "!", "ERROR: " + r.getClient().getSms(), JOptionPane.WARNING_MESSAGE);
}
else {
int response = JOptionPane.showConfirmDialog(null, r.getClient().getSms() + " is " + r.getClient().getStatus() + "...", "Manually Overide?", JOptionPane.OK_CANCEL_OPTION);
if (response == JOptionPane.OK_OPTION) {
serviceQueue.add(r); }
}
writeData(); }
public static void readData() {
try {
Boolean complete = false;
FileReader reader = new FileReader(f);
ObjectInputStream in = xstream.createObjectInputStream(reader);
serviceQueue.clear();
while(complete != true) {
ServiceRequest test = (ServiceRequest)in.readObject();
if(test != null && test.getDate().isAfter(LocalDate.now().minusDays(180))) {
serviceQueue.add(test); }
else { complete = true; }
}
in.close(); }
catch (IOException | ClassNotFoundException e) { e.printStackTrace(); }
}
public static void writeData() {
if(serviceQueue.isEmpty()) { serviceQueue.add(new ServiceRequest()); }
try {
FileWriter writer = new FileWriter(f);
ObjectOutputStream out = xstream.createObjectOutputStream(writer);
for(ServiceRequest r : serviceQueue) { out.writeObject(r); }
out.writeObject(null);
out.close(); }
catch (IOException e) { e.printStackTrace(); }
}
EDIT
The changes cause the concurrent mod to trigger every time rather than intermittently, which I guess means the removal code is better but the error now triggers at it.remove();
public static void removeData(ServiceRequest r) {
readData();
for(Iterator<ServiceRequest> it = serviceQueue.iterator(); it.hasNext();) {
ServiceRequest s = it.next();
if (
s.getClient().getSms() == r.getClient().getSms() &&
s.getTech().getName().equals(r.getTech().getName()) &&
s.getDate().equals(r.getDate())) {
JOptionPane.showMessageDialog(null, s.getClient().getSms() + "'s Service Request with " + s.getTech().getName() + " on " + s.getDate().toString() + " has been removed!");
it.remove(); //Triggers here (line 195)
System.out.print("targetted"); }
}
writeData(); }
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificatio
nException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
at java.util.ArrayList$Itr.next(ArrayList.java:851)
at data.ServiceRequest.removeData(ServiceRequest.java:195)
at data.ServiceRequest.addData(ServiceRequest.java:209) <...>
EDIT
After some more searching, I've switch the for loop to:
Iterator<ServiceRequest> it = serviceQueue.iterator();
while(it.hasNext()) {
and it's back to intermittently triggering. By that I mean the first time I attempt to import data (the removeData method is being triggered from the addData method) it triggers the concurrent mod exception, but the next try it pushes past the failure and moves on to another file. I know there's a lot of these concurrent mod questions, but I'm not finding anything that helps in my situation so links to other answers are more than welcome...

This is not how to do it, to remove elements while going through a List you use an iterator. Like that :
List<ServiceRequest> targets = new ArrayList<ServiceRequest>();
for(Iterator<ServiceRequest> it = targets.iterator(); it.hasNext();) {
ServiceRequest currentServReq = it.next();
if(someCondition) {
it.remove();
}
}
And you will not get ConcurrentModificationException this way if you only have one thread.
If there is multiple threads involved in your code, you may still get ConcurrentModificationException. One way to solve this, is to use Collections.synchronizedCollection(...) on your collection (serviceQueue) and as a result you will get a synchronized collection that will not produce ConcurrentModificationException. But, you code may become very slow.

Put multiple items into DynamoDB by Java code

I would like use batchWriteItem method of SDK Amazon to put a lot of items into table.
I retrive the items from Kinesis, ad it has a lot of shard.
I used this method for one item:
public static void addSingleRecord(Item thingRecord) {
// Add an item
try
{
DynamoDB dynamo = new DynamoDB(dynamoDB);
Table table = dynamo.getTable(dataTable);
table.putItem(thingRecord);
} catch (AmazonServiceException ase) {
System.out.println("addThingsData request "
+ "to AWS was rejected with an error response for some reason.");
System.out.println("Error Message: " + ase.getMessage());
System.out.println("HTTP Status Code: " + ase.getStatusCode());
System.out.println("AWS Error Code: " + ase.getErrorCode());
System.out.println("Error Type: " + ase.getErrorType());
System.out.println("Request ID: " + ase.getRequestId());
} catch (AmazonClientException ace) {
System.out.println("addThingsData - Caught an AmazonClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with AWS, "
+ "such as not being able to access the network.");
System.out.println("Error Message: " + ace.getMessage());
}
}
public static void addThings(String thingDatum) {
Item itemJ2;
itemJ2 = Item.fromJSON(thingDatum);
addSingleRecord(itemJ2);
}
The item is passed from:
private void processSingleRecord(Record record) {
// TODO Add your own record processing logic here
String data = null;
try {
// For this app, we interpret the payload as UTF-8 chars.
data = decoder.decode(record.getData()).toString();
System.out.println("**processSingleRecord - data " + data);
AmazonDynamoDBSample.addThings(data);
} catch (NumberFormatException e) {
LOG.info("Record does not match sample record format. Ignoring record with data; " + data);
} catch (CharacterCodingException e) {
LOG.error("Malformed data: " + data, e);
}
}
Now if i want to put a lot of record, I will use:
public static void writeMultipleItemsBatchWrite(Item thingRecord) {
try {
dataTableWriteItems.addItemToPut(thingRecord);
System.out.println("Making the request.");
BatchWriteItemOutcome outcome = dynamo.batchWriteItem(dataTableWriteItems);
do {
// Check for unprocessed keys which could happen if you exceed provisioned throughput
Map<String, List<WriteRequest>> unprocessedItems = outcome.getUnprocessedItems();
if (outcome.getUnprocessedItems().size() == 0) {
System.out.println("No unprocessed items found");
} else {
System.out.println("Retrieving the unprocessed items");
outcome = dynamo.batchWriteItemUnprocessed(unprocessedItems);
}
} while (outcome.getUnprocessedItems().size() > 0);
} catch (Exception e) {
System.err.println("Failed to retrieve items: ");
e.printStackTrace(System.err);
}
}
but how can I send the last group? because I send only when I have 25 items, but at the end the number is lower.

You can write items to your DynamoDB table one at a time using the Document SDK in a Lambda function attached to your Kinesis Stream using PutItem or UpdateItem. This way, you can react to Stream Records as they appear in the Stream without worrying about whether there are any more records to process. Behind the scenes, BatchWriteItem consumes the same amount of write capacity units as the corresponding PutItem calls. A BatchWriteItem will be as latent as the PUT in the batch that takes the longest. Therefore, using BatchWriteItem, you may experience higher average latency than with parallel PutItem/UpdateItem calls.