I have implemented a Steaming API for twitter. I get the streams perfectly. However, My program never ends. I have tried many combinations but can't figure out why. I am suing Apache AsyncHttpClient in java. My goal is to start the stream for example for 10 seconds, get the streams, and gracefully close the stream and exit the application (I am expecting this to happen when my Main method ends naturally). This is the code below:
public static void main(String[] args) throws Exception
{
TwitterStreamingHttpClient client = new TwitterStreamingHttpClient();
Executor ex = Executors.newSingleThreadExecutor();
ex.execute(client);
Thread.sleep(5000);
client.ceaseStream();
LOG.debug("Keeps running");
}
and this:
public class TwitterStreamingHttpClient extends DefaultHttpAsyncClient implements Runnable
{
private final static Logger LOG = LoggerFactory.getLogger(TwitterStreamingHttpClient.class);
/**
* #throws IOReactorException
*/
public TwitterStreamingHttpClient() throws IOReactorException
{
super();
// TODO: parametrize it, load from config file, spring config file?
this.getCredentialsProvider().setCredentials(new AuthScope("stream.twitter.com", 80),
new UsernamePasswordCredentials("username", "password"));
this.start();
}
public void initiateStream() throws UnsupportedEncodingException, InterruptedException, ExecutionException
{
String requestContent = new String();
requestContent = "track=NothingFeelsBetterThan";
Future future = this.execute(HttpAsyncMethods.createPost(
"https://stream.twitter.com/1.1/statuses/filter.json", requestContent,
ContentType.APPLICATION_FORM_URLENCODED), new TwitConsumer(), null);
Boolean result = future.get();
if(result==null)
{
LOG.error("Requested to close stream!");
return;
}
}
public void ceaseStream()
{
try
{
this.shutdown();
LOG.info("Shutting down the stream");
}
catch (InterruptedException e)
{
LOG.debug("InterruptedException {}", e);
}
}
/*
* (non-Javadoc)
*
* #see java.lang.Runnable#run()
*/
public void run()
{
Thread.currentThread().setName("initiateSTream Thread");
try
{
initiateStream();
Thread.currentThread().interrupt();
}
catch (UnsupportedEncodingException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (ExecutionException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I tried to add a return whereever I though it mightbe helpful. but no luck. Can someone help me with this?
Edit 1: When I use the debug mode, I can see that the "initiateSTream Thread" thread. is still running while the main thread is gone!
Edit 2 (Solution): In the main method, I replaced:
Executor ex = Executors.newSingleThreadExecutor();
ex.execute(client);
with:
Thread thread = new Thread(client);
thread.start();
Now my programs ends after the designated time of streaming. But why? What is the difference between the two approaches?!
Related
I am exploring java.util.concurrent.*
Calculating the square and waiting using Thread.sleep(5000) , the program works as expected, but never terminates.
The red square in eclipse is "ON", that we usually use to terminate the program.
Can you please help in understanding why the program doesn't terminate on completion??
public static void main(String[] args) throws InterruptedException, ExecutionException {
// TODO Auto-generated method stub
try {
SquareCalculator sqC = new SquareCalculator();
sqC.display(1);
Future<Integer> result = sqC.calculate(5);
while(!result.isDone())
{
System.out.println("Waiting for the calculation");
Thread.sleep(1000);
//result.cancel(true);
}
Integer square = result.get();
System.out.println(square);
}catch(Exception e)
{
e.printStackTrace();
System.out.println("Calclulation was interrupted");
}
}
public class SquareCalculator {
private ExecutorService ex = Executors.newSingleThreadExecutor();
public void display(int i) {
// TODO Auto-generated method stub
System.out.println(i);
}
public Future<Integer> calculate(Integer inp)
{
try {
System.out.println("Before sending request");
Future<Integer> res = ex.submit(()->{
Thread.sleep(5000);
return inp*inp;
});
System.out.println("Request sent to caluclate and waiting for the result");
return res;
}catch(Exception e)
{
System.out.println("calculation was interrupted");
return null;
}
//return ex.submit(()->squareing(inp));
}
}
OUTPUT
1
Before sending request
Request sent to caluclate and waiting for the result
Waiting for the calculation
Waiting for the calculation
Waiting for the calculation
Waiting for the calculation
Waiting for the calculation
25
You need to refactor your code and return the object instead of Future. You should also shutdown executor when you are done.
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class SquareCalculator {
private ExecutorService ex = Executors.newSingleThreadExecutor();
public void display(int i) {
// TODO Auto-generated method stub
System.out.println(i);
}
public Integer calculate(Integer inp) {
Integer result;
try {
System.out.println("Before sending request");
Future<Integer> res = ex.submit(() -> {
Thread.sleep(5000);
return inp * inp;
});
System.out.println("Request sent to caluclate and waiting for the result");
result = res.get();
ex.shutdown();
return result;
} catch (Exception e) {
System.out.println("calculation was interrupted");
return null;
}
//return ex.submit(()->squareing(inp));
}
public static void main(String[] args) throws InterruptedException,
ExecutionException {
// TODO Auto-generated method stub
try {
SquareCalculator sqC = new SquareCalculator();
sqC.display(1);
Integer result = sqC.calculate(5);
System.out.println(result);
} catch (Exception e) {
e.printStackTrace();
System.out.println("Calclulation was interrupted");
}
}
}
I would rather create an executor outside the Calculator class and the pass it in the constructor.
This way the application has control over the ExecutorService and shut it down when necessary.
Also, if you create more then one instance of a calculator, all instance use the same executor service, so you can control how many instance can run in parallel.
Blocking in the calculate method works, but defeats the purpose of using another thread to make an async calculation.
public static void main(String[] args) {
// The executor is created by the application and then
// passed to the calculator
ExecutorService executor = Executors.newCachedThreadPool();
SquareCalculator calculator = new SquareCalculator(executor);
// calculate does not block
Future<Integer> calculate = calculator.calculate(12);
try {
while(true) {
try {
// wait a limited amount of time for the computation to complete
Integer result = calculate.get(1, TimeUnit.SECONDS);
System.out.println(result);
if(calculate.isDone()) {
// If the computation was either complete or cancelled just quit
break;
}
} catch (TimeoutException e) {
// We expect timeouts so we don't quit the loop for them
System.out.println("Waiting for result");
}
}
} catch (InterruptedException | ExecutionException e) {
// If there was an error or the computation was interrupted just quit.
e.printStackTrace();
}
// Shut down the executor so we do not leak pools.
executor.shutdown();
}
public class SquareCalculator {
private ExecutorService ex;
public SquareCalculator(ExecutorService ex) {
super();
this.ex = ex;
}
public void display(int i) {
System.out.println(i);
}
public Future<Integer> calculate(Integer inp) {
try {
System.out.println("Before sending request");
Future<Integer> res = ex.submit(() -> {
Thread.sleep(5000);
return inp * inp;
});
System.out.println("Request sent to caluclate and waiting for the result");
return res;
} catch (Exception e) {
System.out.println("calculation was interrupted");
return null;
}
}
}
If you want the VM to shut down, call System.exit(). Yes, the VM can automatically close without calling that method as well; it does this if ALL still 'live' threads have the 'daemon' flag up (the Thread class has a .setDaemon method for this purpose), but that's bad code style. If the point is to shut down, then shut down (with System.exit).
Specifically here, the threads created by Executors.newSingleThreadExecutor(); aren't marked as daemon threads. You can fix that by supplying a thread creator to the call.
But, really, don't. Use System.exit.
I have 3 rfid readers where I am reading rfid tags. I am running those but for some reason I am only receiving info from the last reader(thread) that is being processed. What am I missing? The readers are in an array containing the ipaddress, username, port, password. Can I listen on the same service for all of them? I'm new to threads........TagInventory is the name of the class where all of this is located.
Here is the code:
private void Start() throws AlienReaderException, IOException{
ThreadStop = false;
service= new MessageListenerService(3900);
service.setMessageListener(this);
service.startService();
System.out.println("length of readers: "+Reader.ipAddress.length);
for (lastThreadId = 0; lastThreadId < Reader.ipAddress.length; lastThreadId++)
{
m_inventory[lastThreadId] = new AlienReader(Reader.ipAddress[lastThreadId], Reader.port, Reader.username[lastThreadId], Reader.password[lastThreadId]);
log.info("taginventory reader: "+ Reader.ipAddress[lastThreadId]+"Thread: "+lastThreadId);
m_run_process[lastThreadId] = new Thread(new StartInventoryThread(Reader.ipAddress[lastThreadId], Reader.port, Reader.username[lastThreadId], Reader.password[lastThreadId], m_inventory[lastThreadId]));
m_run_process[lastThreadId].start();
}
--lastThreadId;
try
{
// Thread.sleep(1000);
Thread.sleep(2000);
}
catch (Exception ex)
{
ex.getMessage();
}
}
class StartInventoryThread implements Runnable{
private String ip;
private int port;
private String user;
private String pwd;
private AlienReader ar;
StartInventoryThread(String ip, int port, String user, String pwd, AlienReader ar){
this.ip=ip;
this.port=port;
this.user=user;
this.pwd=pwd;
this.ar=ar;
}
#Override
public void run() {
try {
while(!stopInventory){
startRead(ip,port,user,pwd);
}
} catch (AlienReaderException | InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public void startRead(String ip, int port, String user, String password) throws AlienReaderException, InterruptedException, UnknownHostException{
String myIP=InetAddress.getLocalHost().getHostAddress();
//System.out.println("ip"+ ip);
AlienReader ar= new AlienReader(ip, port, user, password);
ar.open();
//log.info("Reader" + ar.getIPAddress());
ar.setNotifyAddress(myIP, 3900);
ar.setNotifyFormat(AlienClass1Reader.TEXT_FORMAT);
//ar.setNotifyTrigger("TrueFalse");
ar.setNotifyTrigger("Add");
ar.setNotifyMode(AlienClass1Reader.ON);
// log.info("MessageListenerService has started for reader: " + ip);
//complete process in here
ar.autoModeReset();
ar.setAutoStopTimer(5000); // Read for 5 seconds
ar.setAutoMode(AlienClass1Reader.ON);
tagTable.setTagTableListener(tagTableListener);
tagTable.setPersistTime(3600);
//tagTable.setPersistTime(1800000);
ar.close();
long runTime = 10000; // milliseconds
long startTime = System.currentTimeMillis();
do {
Thread.sleep(1000);
} while(service.isRunning()
&& (System.currentTimeMillis()-startTime) < runTime);
// Reconnect to the reader and turn off AutoMode and TagStreamMode.
// log.info("\nResetting Reader");
ar.open();
ar.autoModeReset();
ar.setNotifyMode(AlienClass1Reader.OFF);
ar.close();
}
public static void main(String args[]){
Thread thr=new Thread(new Runnable(){
#Override
public void run(){
try {
new TagInventory();
} catch (AlienReaderException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
thr.start();
}
Overview
I see a number of issues and potential hazards in your logic throughout the code. I will step through each and guide you to the corresponding concepts that align with these issues.
Main
In your main(String args[]), add a join() call so that the calling thread (likely the main thread) can wait until Thread thr finishes before exiting.
public static void main(String args[]){
Thread thr=new Thread(new Runnable(){
#Override
public void run(){
try {
new TagInventory();
} catch (AlienReaderException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
thr.start();
thr.join();
}
Start
Assuming you are expecting the threads to finish by the time the Start() function finishes I will explain how to obtain that functionality. Do not utilize Thread.sleep() as it is considered poor programming. Instead utilize the join() function. The forces the calling thread to wait or block until that thread has finished. The logic added below will cycle through each of the m_run_process and block until it has finished. If you have another location where you want to ensure they all finish, you can put the for-loop there instead.
private void Start() throws AlienReaderException, IOException{
ThreadStop = false;
service= new MessageListenerService(3900);
service.setMessageListener(this);
service.startService();
System.out.println("length of readers: "+Reader.ipAddress.length);
for (lastThreadId = 0; lastThreadId < Reader.ipAddress.length; lastThreadId++)
{
m_inventory[lastThreadId] = new AlienReader(Reader.ipAddress[lastThreadId], Reader.port, Reader.username[lastThreadId], Reader.password[lastThreadId]);
log.info("taginventory reader: "+ Reader.ipAddress[lastThreadId]+"Thread: "+lastThreadId);
m_run_process[lastThreadId] = new Thread(new StartInventoryThread(Reader.ipAddress[lastThreadId], Reader.port, Reader.username[lastThreadId], Reader.password[lastThreadId], m_inventory[lastThreadId]));
m_run_process[lastThreadId].start();
}
--lastThreadId;
for(Thread inventoryThread : m_run_process)
inventoryThread.join()
}
StartRead
There seems to be a lot of problems here where you are accesses non-thread-safe objects and variables; service, stopInventory, and tagTable (without seeing the full file I am making assumptions and I do not see synchronization anywhere). Now, I recommend reading up on the Java Memory Model, Java Memory Barrier, Atomic Operations, Synchronization, Thread-Safety, and the JVM. In essence, your logic can cause overwriting of data and/or threads see different caches values. To rectify you must make shared, mutable instances thread-safe through synchronization and atomicity.
public void startRead(String ip, int port, String user, String password) throws AlienReaderException, InterruptedException, UnknownHostException{
String myIP=InetAddress.getLocalHost().getHostAddress();
//System.out.println("ip"+ ip);
AlienReader ar= new AlienReader(ip, port, user, password);
ar.open();
//log.info("Reader" + ar.getIPAddress());
ar.setNotifyAddress(myIP, 3900);
ar.setNotifyFormat(AlienClass1Reader.TEXT_FORMAT);
//ar.setNotifyTrigger("TrueFalse");
ar.setNotifyTrigger("Add");
ar.setNotifyMode(AlienClass1Reader.ON);
// log.info("MessageListenerService has started for reader: " + ip);
//complete process in here
ar.autoModeReset();
ar.setAutoStopTimer(5000); // Read for 5 seconds
ar.setAutoMode(AlienClass1Reader.ON);
tagTable.setTagTableListener(tagTableListener);
tagTable.setPersistTime(3600);
//tagTable.setPersistTime(1800000);
ar.close();
long runTime = 10000; // milliseconds
long startTime = System.currentTimeMillis();
do {
Thread.sleep(1000);
} while(service.isRunning()
&& (System.currentTimeMillis()-startTime) < runTime);
// Reconnect to the reader and turn off AutoMode and TagStreamMode.
// log.info("\nResetting Reader");
ar.open();
ar.autoModeReset();
ar.setNotifyMode(AlienClass1Reader.OFF);
ar.close();
}
I have simple java method to upload file from local disk to remote web service, and it's working great in controller class.
Now I moved this method into separated class. I try to call this method from thread :
public void run(){
while (true){
System.out.println("Geting next Task for processing ...");
try {
CommonUtils.uploadVideo(32);
Thread.sleep(20000);
} catch (InterruptedException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Inject
static
WSClient ws;
synchronized static void uploadVideo(int taskId) throws InterruptedException, ExecutionException, IOException {
....
....
...
com.ning.http.client.AsyncHttpClient underlyingClient = (com.ning.http.client.AsyncHttpClient) ws.getUnderlying();
final String response;
response = underlyingClient.preparePost("http://192.16.10.125:9001/publish").
setHeader("watermark", "0").
addBodyPart(new FilePart("file", file)).execute().get().getResponseBody().toString();
}
And I got an error :
Exception in thread "Thread-8" java.lang.NullPointerException
at utils.CommonUtils.uploadVideo(CommonUtils.java:308)
at utils.TaskRunner.run(TaskRunner.java:88)
at java.lang.Thread.run(Unknown Source)
What is wrong? Is the problem Asynchronous call web service?
I am working on a java application in which I am facing a problem. When I send a file to a server and an exception is thrown, the file is not sent. How can I retry sending the file?
public void uploadtxtFile(String localFileFullName, String fileName, String hostDir)
throws Exception {
File file = new File(localFileFullName);
if (!(file.isDirectory())) {
if (file.exists()) {
FileInputStream input = null;
try {
input = new FileInputStream(new File(localFileFullName));
if (input != null) {
hostDir = hostDir.replaceAll("//", "/");
logger.info("uploading host dir : " + hostDir);
//new
// TestThread testThread=new TestThread(hostDir,input);
// Thread t=new Thread(testThread);
//
// try{
// t.start();
//
// }catch(Exception ex){
// logger.error("UPLOADE start thread create exception new:" + ex);
// }
// // new end
DBConnection.getFTPConnection().enterLocalPassiveMode();
// the below line exeption is come
boolean bool = DBConnection.getFTPConnection().storeFile(hostDir, input);
//input.close();//new comment
if (bool) {
logger.info("Success uploading file on host dir :"+hostDir);
} else {
logger.error("file not uploaded.");
}
} else {
logger.error("uploading file input null.");
}
}catch(CopyStreamException cs)
{ logger.error("Copy StreamExeption is come "+cs);
} catch(Exception ex)
{
logger.error("Error in connection ="+ex);//this is catch where I handle the exeption
}finally {
// boolean disconnect= DBConnection.disConnect();
input.close();
}
} else {
logger.info("uploading file is not exists.");
}
}
}
This is the code and I want to restart the file uploading but I don't have any idea. I tried it using the thread but the exception is thrown again. I also tried to use a while loop, but it loops infinitely and also shows the exception as well as another exception.
Below is the thread code that I use:
public void run() {
System.out.println("Enter Thread TestThread");
DBConnection.getFTPConnection().enterLocalPassiveMode();
// System.out.println("Error in DBConnection ");
//here server timeout error is get
boolean bool1=false;
boolean bool=true;
try {
bool = DBConnection.getFTPConnection().storeFile(hostDir1, input1);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
//disconnect();
try {
input1.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
if (bool) {
System.out.println("File is Uploded");
} else {
while(bool!=true){
try {
DBConnection.getFTPConnection().enterLocalPassiveMode();
bool1=DBConnection.getFTPConnection().storeFile(hostDir1, input1);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
//disconnect();
try {
input1.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println("file not uploaded."+bool1);
bool=bool1;
}
}
}
}
}
Can any one have a solution to how to upload the file to the server?
The exception is shown below:
Software caused connection abort: recv failed
Software caused connection abort: socket write error
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
Add a static class as below in a class from where you are calling the method which need to be retried:
static class RetryOnExceptionStrategy {
public static final int DEFAULT_RETRIES = 3;
public static final long DEFAULT_WAIT_TIME_IN_MILLI = 2000;
private int numberOfRetries;
private int numberOfTriesLeft;
private long timeToWait;
public RetryOnExceptionStrategy() {
this(DEFAULT_RETRIES, DEFAULT_WAIT_TIME_IN_MILLI);
}
public RetryOnExceptionStrategy(int numberOfRetries,
long timeToWait) {
this.numberOfRetries = numberOfRetries;
numberOfTriesLeft = numberOfRetries;
this.timeToWait = timeToWait;
}
/**
* #return true if there are tries left
*/
public boolean shouldRetry() {
return numberOfTriesLeft > 0;
}
public void errorOccured() throws Exception {
numberOfTriesLeft--;
if (!shouldRetry()) {
throw new Exception("Retry Failed: Total " + numberOfRetries
+ " attempts made at interval " + getTimeToWait()
+ "ms");
}
waitUntilNextTry();
}
public long getTimeToWait() {
return timeToWait;
}
private void waitUntilNextTry() {
try {
Thread.sleep(getTimeToWait());
} catch (InterruptedException ignored) {
}
}
}
Now wrap your method call as below in a while loop :
RetryOnExceptionStrategy errorStrategy=new RetryOnExceptionStrategy();
while(errorStrategy.shouldRetry()){
try{
//Method Call
}
catch(Exception excep){
errorStrategy.errorOccured();
}
}
Basically you are just wrapping you method call in while loop which will
keep returnig true till your retry count is reached to zero say you started with 3.
Every time an exception occurred, the exception is caught and a method is called
which will decrement your retryCount and method call is again executed with some delay.
A general way of working with such application is:
Create a class, say, UploadWorker which extends Callable as the wrapper. Make the wrapper return any error and detail information you need when it fails.
Create a ExecutorService (basically a thread pool) for this wrapper to run in threads.
Submit your UploadWorker instance and then you get a Future. Call get() on the future to wait in blocking way or simply wait some time for the result.
In case the get() returns you the error message, submit your worker again to the thread pool.
I am facing a strange programming problem. It has exhausted me but with no solution found!
My program is mainly dependent on an event (Java Listener) that is fired from an external hardware for receiving an audio message from that hardware.Inside the eventHandler i do the following
pass the received message to a static method "decode" from another class which returns data
then I open FileOutputStream, write these data "audio" to a file,and close the FileOutputStream.
I call a static method "play" from another class to play the audio file.
The problem is: whenever the method "Play" is called for the first time, it executes correctly but it causes the event to stop raising and the program to terminate but without exceptions. When I comment out the play method, everything becomes okay!
Do you have some idea about a method causing program termination ?
public void messageReceived(int to,Message message)
{
speexAudioMsg msg = (speexAudioMsg)message;
try{
byte[] output = jspeexDecoder.decode(msg.get_frame());
os = new FileOutputStream(file);
os.write(output);
os.close();
Player.play();
}
catch (IOException ex) {ex.printStackTrace();}
}
You are probably using the event thread to play the music. Try calling Player.play() in a new thread.
new Thread(new Runnable() { public void run() {Player.play()}}).start();
here is an example:
static String url = "http://www.stackoverload.com";
public static void threadTest() {
new Thread(new Runnable() {
#Override
public void run() {
// TODO Auto-generated method stub
try {
URL url2 = new URL(url);
url2.openStream();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}).run();