So I'm writing a video playback library, the details aren't important. What's going on is this bit of code needs to run at the end of the run() method of the decoder thread:
System.out.println("Video decoding complete");
int a = 0, b = 0;
do
{
a = pictures.getCount();
b = samples.getCount();
}while(a > 0 || b > 0);
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
The problem is, anything passed the do{}while doesn't execute. Here's the weird part, this bit of code executes when the System.out.println is added in the while loop:
System.out.println("Video decoding complete");
int a = 0, b = 0;
do
{
System.out.println("Waiting for packets to drain.");
a = pictures.getCount();
b = samples.getCount();
}while(a > 0 || b > 0);
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
I suspect that the compiler knows I'm trying to get it to run a loop doing nothing for a bit, and it just snips out the code or something. But really I have no idea what's happening. If someone knows better than I, I'd love to have a better solution. Such a simple thing that I'm hung up on here!
My guess is that pictures.getCount() and samples.getCount() read non-volatile fields. When you only read a non-volatile field it can be inlined for performance reasons, however if you do something like call a synchronized method (And System.out is synchronized) it doesn't optimise the code this way and has to perform the look up each time.
I suggest you try adding an empty synchronized block and see if this still works, i.e. do this instead of the println
synchronized(this) { }
I ended up doing this as per #slim's advice:
System.out.println("Video decoding complete");
this.decoderComplete = true;
//wait until notified that packets are done draining
synchronized(this)
{
try {
this.wait();
} catch (InterruptedException e) {
videoComplete = true;
this.container.close();
e.printStackTrace();
}
}
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
this.container.close();
And in the other thread, once we know we're done reading all the packets:
if(this.packetHandlerRunnable.getNumAudioPackets() <= 0
&& this.packetHandlerRunnable.getNumVideoPackets() <= 0
&& this.packetHandlerRunnable.isDecoderComplete())
{
synchronized(packetHandlerRunnable)
{
this.packetHandlerRunnable.notify();
}
}
Related
I have a problem with my Java socket and connection. It seems like my input stream does not read any data even though there is data on the stream. Here is an example on how we read data:
InputStream is = socket.getInputStream();
StringBuffer buf = new StringBuffer();
final int SO_TIME = 2500;
socket.setSoTimeout(SO_TIME);
long readTime = 0;
boolean remoteSocketClosed = false;
//Read data while the Remote-Socket is opened and the timeout is OK
while (!remoteSocketClosed && readTime < 30000)
{
try
{
int c = is.read();
if (c != -1)
{
buf.append((char)c);
}
else //c==-1
{
remoteSocketClosed = true;
}
}
catch (SocketTimeoutException socketTimeout)
{
readTime += SO_TIME;
}
}
if (readTime >= 30000)
{
throw new IOException("No answer from server after request (" + readTime +"ms )");
}
else
{
if (buf.length() > 0)
{
return buf;
}
else
{
return null;
}
}
finally
{
try
{
if (socket!=null)
socket.close();
}
catch (Exception e)
{
l.error("Socket not closed", e);
}
try{Thread.sleep(250);}catch(Exception e){}
}
Now this looks fine to me. However when there is data on the stream, it will not get picked up. I can use telnet and see that data is coming in on that connection, yet Java just throws the IOException (timeout).
The server is running on the same machine (localhost), and we can send data to the server, yet the answer is lost somehow (only in Java not in telnet). Is this some weird setting I don't know about? We are using Windows 10 and Java 8, firewalls are all disabled. I have never experienced something like this so any help is appreciated.
Sometimes you just don't see the forest for the trees and have to sleep over it...
The problem is my condition
if (c != -1)
{
buf.append((char)c);
}
else //c==-1
{
remoteSocketClosed = true;
}
But the server never closes the connection... So I never exit my loop and im stuck for 30 seconds running into socket timeouts. After the 30 seconds of waiting my answer is in the StringBuffer but of course my readTime is 30000 so I throw the exception rather providing an answer... I will have to think of an EndOfPackage Byte or something to avoid this.
Thanks for any help tho
Thank you for all your help. I tried everything from creating a new Thread. To changing around the way I use the writer.flush() and the writer.newLine(). It seems like the code keeps getting hung up at the while loop, constantly running the Thread.sleep() nonstop. I cannot provide the batch file because it is sensitive information, but the String command variable on top of the code is the path I am using to access the command. Please if you do answer this question please run the code first with test bat file and two input field.
Batch Script:
#ECHO OFF
SET /P _inputname= Please enter an name:
SET /P _inputpassword= Please enter an password:
IF "%_inputpassword%"=="1234" GOTO :they_said_1234
ECHO You entered the wrong password!
pause
GOTO
:they_said_1234
ECHO You entered 1,2,3,4!
pause
Java Code:
import java.io.*;
import java.nio.CharBuffer;
import java.util.ArrayList;
public class command{
public static void main(String[] args) {
//String command="cmd /c d: && cd UPSDATA\\Virtualization Scripts\\EMC ESXi Grab\\EMC-ESXi-GRAB-1.3.7 && GRAB_RUN ";
//String command="cmd /c date";
String command = "cmd /c cd C:\\Users\\HFB2VZN\\Desktop\\folderG";
try {
Process process = Runtime.getRuntime().exec(command);
try (Writer writer = new OutputStreamWriter(process.getOutputStream());
Reader reader = new InputStreamReader(process.getInputStream())) {
CharBuffer buf = CharBuffer.allocate(80);
int tries = 2;
while (process.isAlive()) {
while (reader.ready() && reader.read(buf) > 0) {
//3
System.out.println("buf.flip() ran");
System.out.append(buf.flip());
buf.clear();
}
if (tries-- == 0) {
System.out.println("Destroyed");
process.destroy();
break;
}
//1
writer.write("random");
writer.flush();
while (!reader.ready()) {
//2
System.out.println("while() loop, Thread runs in non stop loop");
Thread.sleep(800);
}
}
}
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
}`
Compared to this example, executing a batch file will be much slower and having multiple commands may imply that there’s no available output for some time before the next output is generated. Not considering this may cause the loops get out of sync and after the process terminated, you must not execute the while (!reader.ready()) {…} loop without checking whether the process is still alive.
Since your batch file’s second GOTO lacks a target (it’s likely supposed to branch backwards), this batch file may terminal earlier than intended. Since no-one reads the error channel, this stays unnoticed. That could be the reason for hanging in that loop. Note further, that you are generating one input to the batch file per loop iteration, but have limited the number of iteration to 2 per tries variable. For a batch file expecting three inputs (name, password, pause), that’s too little.
The main problem is, there is no way to detect whether a subprocess is actually waiting for our input. This is what we have to work-around here. But a temporary stopping of producing output does not always imply that the program now waits for input.
I fixed your last GOTO to jump to the beginning of the batch file and used the following code to make two attempts entering the right password on the second.
try {
Process process = Runtime.getRuntime().exec(command);
try(Writer writer = new OutputStreamWriter(process.getOutputStream());
Reader reader = new InputStreamReader(process.getInputStream())) {
CharBuffer buf = CharBuffer.allocate(80);
int tries = 2;
while(process.isAlive()) {
do {
if(!buf.hasRemaining()) {
buf.flip();
buf = CharBuffer.allocate(buf.capacity()*2).put(buf);
}
do {} while(reader.ready() && reader.read(buf) > 0);
if(buf.position() > 0) {
char c = buf.get(buf.position()-1);
if(c==':' || c=='.') break;
}
long deadLine = System.nanoTime() + TimeUnit.SECONDS.toNanos(1);
for(long remaining = 1;
!reader.ready() && process.isAlive() && remaining > 0;
remaining = deadLine - System.nanoTime()) {
LockSupport.parkNanos(Math.min(
TimeUnit.MILLISECONDS.toNanos(100),
remaining));
}
} while(reader.ready());
String input = buf.flip().toString();
buf.clear();
System.out.print(input);
String value;
if(input.endsWith("name:")) value = "aName";
else if(input.endsWith("password:")) value = tries>1? "wrongPassword": "1234";
else {
value = "<any key>";
tries--;
}
System.out.println("<- "+value);
writer.write(value);
writer.flush();
if(tries == 0) {
System.out.println("Destroying");
process.destroy();
break;
}
}
}
} catch(IOException e) {
e.printStackTrace();
}
Of course, if you can’t fix the GOTO statement, you have to provide the right password at the first attempt.
The code above will wait up to one second for the availability of new output, unless it recognizes one of the expected prompts in the output. And it won’t wait when the process is not alive anymore.
I have some old code I am working with, and I'm not too experienced with Threads (mostly work on the front end). Anyway, this Thread.sleep is causing the thread to hang and I'm unsure what to do about it. I thought about using a counter and throwing a Thread.currentThread.interupt, but unsure of where to put it or which thread it will interupt. Here is an example of the dump. As you can see the thread count is getting pretty high at 1708.
Any advice?
"Thread-1708" prio=6 tid=0x2ceec400 nid=0x2018 waiting on condition
[0x36cdf000] java.lang.Thread.State: TIMED_WAITING (sleeping) at
java.lang.Thread.sleep(Native Method) Locked ownable synchronizers:
- None "Thread-1707" prio=6 tid=0x2d16b800 nid=0x215c waiting on condition [0x36c8f000] java.lang.Thread.State: TIMED_WAITING
(sleeping) at java.lang.Thread.sleep(Native Method) Locked ownable
synchronizers:
- None
#Override
public void run()
{
Connection con = null;
int i = 0;
while (is_running)
{
try
{
con = ConnectionManager.getConnection();
while (!stack.isEmpty())
{
COUNT++;
String line = (String) stack.pop();
getPartMfr(line);
try
{
if (this.mfr != null && !this.mfr.equals(EMPTY_STR))
{
lookupPart(con, line);
}
}
catch (SQLException e)
{
e.printStackTrace();
}
if (COUNT % 1000 == 0)
{
Log log = LogFactory.getLog(this.getClass());
log.info("Processing Count: " + COUNT);
}
}
}
catch (NamingException e)
{
e.printStackTrace();
}
catch (SQLException e)
{
e.printStackTrace();
}
finally
{
try
{
ConnectionManager.close(con);
}
catch (SQLException e)
{
e.printStackTrace();
}
}
try {
Thread.sleep(80);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
this.finished = true;
}
Here is where it calls the run method, as you can see it does set it to false, but I guess it is missing threads?
HarrisWorker w[] = new HarrisWorker[WORKER_POOL_SIZE];
try
{
for (int i = 0; i < w.length; i++)
{
w[i] = new HarrisWorker(pw);
w[i].start();
}
pw.println(headers());
File inputDir = new File(HARRIS_BASE);
String files[] = inputDir.list();
for (String file : files)
{
try
{
File f = new File(HARRIS_BASE + File.separator + file);
if (f.isDirectory())
continue;
final String workFile = workDir + File.separator + file;
f.renameTo(new File(workFile));
FileReader fr = new FileReader(workFile);
BufferedReader br = new BufferedReader(fr);
String line = br.readLine();
boolean firstLine = true;
while (line != null)
{
if (firstLine)
{
firstLine = false;
line = br.readLine();
continue;
}
if (line.startsWith(","))
{
line = br.readLine();
continue;
}
// if(line.indexOf("103327-1") == -1)
// {
// line = br.readLine();
// continue;
// }
HarrisWorker.stack.push(line);
line = br.readLine();
}
br.close();
fr.close();
for (int i = 0; i < w.length; i++)
{
w[i].is_running = false;
while (!w[i].finished)
{
Thread.sleep(80);
}
}
move2Processed(file, workFile);
long etime = System.currentTimeMillis();
System.out.println("UNIQUE PARTS TOTAL FOUND: " + HarrisWorker.getFoundCount() + " of " + HarrisWorker.getUniqueCount() + ", "
+ (HarrisWorker.getFoundCount() / HarrisWorker.getUniqueCount()));
System.out.println("Time: " + (etime - time));
}
catch (Exception e)
{
e.printStackTrace();
File f = new File(workDir + File.separator + file);
if (f.exists())
{
f.renameTo(new File(HARRIS_BASE + File.separator + ERROR + File.separator + file));
}
}
}
}
As a direct answer to the question in your title - nowhere. There is nowhere in this code that needs a Thread.interrupt().
The fact that the thread name is Thread-1708 does not necessarily mean there are 1708 threads. One can choose arbitrary names for threads. I usually include the name of the executor or service in the thread name. Maybe 1600 are now long stopped and there are only around a hundred alive. Maybe this particular class starts naming at 1700 to distinguish from other uses.
1708 threads may not be a problem. If you have a multi-threaded server that is serving 2000 connections in parallel, then it certainly expectable that there are 2000 threads doing that, along with a bunch of other threads.
You have to understand why the sleep is there and what purpose it serves. It's not there to just hog memory for nothing.
Translating the code to "plaintext" (btw it can be greatly simplified by using try-with-resources to acquire and close the connection):
Acquire a connection
Use the connection to send (I guess) whatever is in the stack
When failed or finished - wait 80ms (THIS is your sleep)
If run flag is still set - repeat from step 1
Finish the thread.
Now reading through this, it's obvious that it's not the sleep that's the problem. It's that the run flag is not set to false. And your thread just continues looping, even if it can't get the connection at all - it will simply spend most of its time waiting for the retry. In fact - even if you completely strip the sleep out (instead of interrupting it mid-way), all you will achieve is that the Threads will start using up more resources. Given that you have both a logger and you print to stdout via printStackTrace, I would say that you have 2 problems:
Something is spawning threads and not stopping them afterwards (not setting their run flag to false when done)
You are likely getting exceptions when getting the Connection, but you never see them in the log.
It might be that the Thread is supposed to set it's own run flag (say when the stack is drained), but you would have to decide that yourself - that depends on a lot of specifics.
Not an answer but some things you should know if you are writing code for a live, production systemn:
:-( Variable and method both have the same name, run. A better name for the variable might be keep_running Or, change the sense of it so that you can write while (! time_to_shut_down) { ... }
:-( Thread.sleep(80) What is this for? It looks like a big red flag to me. You can never fix a concurrency bug by adding a sleep() call to your code. All you can do is make the bug less likely to happen in testing. That means, when the bug finally does bite, it will bite you in the production system.
:-( Your run() method is way too complicated (the keyword try appears four times). Break it up, please.
:-( Ignoring five different exceptions catch (MumbleFoobarException e) { e.printStackTrace(); } Most of those exceptions (but maybe not the InterruptedException) mean that something is wrong. Your program should do something more than just write a message to the standard output.
:-( Writing error messages to standard output. You should be calling log.error(...) so that your application can be configured to send the messages to someplace where somebody might actually see them.
I have two processes that may access the same file concurently and wanted to implement file locking. The trouble seems to be that one process is written in java and the other in C and it is not clear how low level locking is implemented on java side. The platform is Solaris 10. I tried to introduce locking on the file to prevent updates being done by Java process while C process is reading the file. My idea was to attempt to acquire a lock from the java code 10 times and only then to unconditionaly write to the file (I assumed the lock type was an advisory lock). However, the java tryLock() breaks C process' lock on the second attempt and corrupt the reading.
Here is the code, simplified (Java):
int iAttemptCnt = 0;
FileChannel wchannel = new FileOutputStream(new File(fileName), false).getChannel();;
FileLock flock;
while(true){
try{
MyLog.log(MyLog.LVL_INFO, "attempt to lock file");
if( (flock = wChannel.tryLock()) == null ){
// lock held by another program
if(++iAttemptCnt >= 10
break;
}
else{
MyLog.log(MyLog.LVL_INFO, " file locked");
break;
}
}catch(OverlappingFileLockException ofle){
.......
if(++iAttemptCnt >= 10 ){
...
break;
}
}catch(IOException ioe){
throw new IOException("failed to lock the file");
}
try{
MyLog.log(MyLog.LVL_INFO, "File already locked, retrying in one second");
Thread.sleep(1000);
}catch(InterruptedException ie){
.....
}
}
C code uses fcntl:
fd = open(filename, O_RDONLY);
.....
lck.l_type = F_RDLCK;/* F_RDLCK setting a shared or read lock */
lck.l_whence = 0; /* offset l_start from beginning of file */
lck.l_start = 0LL;
lck.l_len = 0LL; /* until the end of the file address space */
....
while( fcntl(fd, F_SETLK64, &lck) < 0){
if( errno == EAGAIN )
....
else if (errno == EIO )
...
else if( errno == ENOLCK)
...
else if (errno == EDEADLK)
...
if(++ii == 10 ){
break;
}
...
sleep(1);
}
MyLongLastingRead();
...
lck.l_type = F_UNLCK;
fcntl(fd, F_SETLK, &lck);
close(fd);
Does tryLock() really checks for lock?
I am not sure if this will solve your problem or not but in the examples that I had seen flock structure's l_pid field was set as below.
fl.l_pid = getpid();
In your question you are not setting this field. Try and see if it makes any difference.
I hope it helps.
I have a JAVA game server that uses 1 thread per TCP connection. (I know it's bad but i'll have to keep it this way for now). On a (3.2Ghz 6cor x2 machine, 24GB RAM, windows server 2003 64bits) and here is a piece of the code:
public void run()
{
try
{
String packet = "";
char charCur[] = new char[1];
while(_in.read(charCur, 0, 1)!=-1 && Server.isRunning)
{
if (charCur[0] != '\u0000' && charCur[0] != '\n' && charCur[0] != '\r')
{
packet += charCur[0];
}else if(!packet.isEmpty())
{
parsePlayerPacket(packet);
packet = "";
}
}
}catch(Exception e)
{
e.printStackTrace();
}
finally
{
try{
kickPlayer();
}catch(Exception e){e.printStackTrace();};
Server.removeIp(_ip);
}
}
After about 12 hours or more of server upTime (and about 3.000 players connected) the server starts eating 100% of all the 12 CPUs for ever, until I manually reboot the JAVA application. So the game starts lagging verry bad and my players starts complaining.
I have tried profiling the application and here is what I came up with:
So I am guessing that the problem is coming from here:
while(_in.read(charCur, 0, 1)!=-1 && Server.isRunning)
knowing that the variable "_in" is a reader of the socket input : (_in = new BufferedReader(new InputStreamReader(_socket.getInputStream()))).
Why on earth _in.read() takes so much CPU after a long server upTime?
I have tried putting a Thread.sleep(1); and more inside the While loop, but doesn't do anything, I guess the problem is inside of the BufferedReader.read() method.
Does anyone have any idea of what can cause this?? And how to fix it?
This is a duplicate of your previous question: An infinite loop somewhere in my code. Please do not open up a new question, but instead use the editing functions.
That being said, 3000 threads is definitely a lot and would most likely cause excessive amounts of context switching. Instead of starting a new thread for each connection, consider using non-blocking IO facilities in Java. Examples can be found here: http://download.oracle.com/javase/1.4.2/docs/guide/nio/example/index.html
I don't know why the call is slow but I would never read one byte at a time in a tight loop. Who knows what kind of overhead the internal function has.
I would read all the data that is available currently in the stream and parse that.
This would require a buffer and some extra bookkeeping but anyway faster than reading byte by byte from a stream.
'1 thread per TCP connection'
'about 3.000 players connected'
= 3.000 threads?!
My guess: the maximum amount of threads that can repeatedly copy one byte at a time is around 3.000. That doesn't sound so weird.
Solution: less threads and read more bytes in one go.
You could use a executorService. There is a simplistic example in the javadoc: http://download.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html
It doesn't look like you ever close the BufferedReader either, unless you are attempting it in the kickPlayer() method.
Each reader may be living a lot longer than you realise.
I'm also stuck on this same problem, I have also tried many solutions but no luck with read(byte). But when I have tried with readLine(), it works well. #Reacen did you found any other answer please let me know too.
public void run() {
try {
InputStream input = clientSocket.getInputStream();
BufferedReader bf = new BufferedReader(new InputStreamReader(input));
while (isRunning) {
if (mainServer.isStopped()) {
disconnect();
}
if (clientSocket.isClosed()) {
isRunning = false;
break;
}
// New Code Receive commands from device
String result = null;
try {
result = bf.readLine();
if (result == null) {
disconnect();
} else {
Pattern pattern = Pattern.compile("(?<=\\[).*(?=\\])");
Matcher matcher = pattern.matcher(result);
if (matcher.find()) {
result = matcher.group(0);
}
}
} catch (SocketTimeoutException e) {
logger.debug("Socket Read Timeout: " + remoteAddress);
} catch (SocketException e) {
isRunning = false;
break;
}
if (result == null || result.trim().length() == 0) {
continue;
}