Continuous image loading/processing & display on button click (in Swing) - java

I have trouble with continuous image loading&processing&display in Swing:
Below there is simple case where clicking a button causes my program to grab frame from webcam and then display this picture in jlabel. It works as it should, but i have to consecutively click this "Start" button in order to get new image(frame) shown.
private void StartActionPerformed(java.awt.event.ActionEvent evt) {
displayed_img = this.getPIC_COLOR(player);
img_field.setIcon(new ImageIcon(displayed_img));
}
Im more demanding and i would like to make some image processing during live video stream, therefore i need to "grab" & "show" my frames continuously. You might say that i can just launch webcam stream, but it is not what i want to achieve since i'm about to implement some threshholding/etc. functions which will modify my image on-fly. Therefore i need to perform this grabbing&processin&display as fast as possible (delay is a no-no since i want to achieve something like 10 fps including image processing). Despite the fact that i delay is utterly undesirable i tried to make some Thread.sleep however it didnt work.
Simple while(true) /* presented below */ does not work, my program "hangs" and not even single frame is displayed, although in debugger it keeps working over and over.
private void StartActionPerformed(java.awt.event.ActionEvent evt) {
while(true){
displayed_img = this.getPIC_COLOR(player);
//threshholding & further processing...
img_field.setIcon(new ImageIcon(displayed_img));
}
}
just in case getPIC_COLOR() was required, i paste it below:
public BufferedImage getPIC_COLOR(Player player){
FrameGrabbingControl frameGrabber = (FrameGrabbingControl)player.getControl("javax.media.control.FrameGrabbingControl");
Buffer buf = frameGrabber.grabFrame();
Image img = (new BufferToImage((VideoFormat)buf.getFormat()).createImage(buf));
BufferedImage buffImg = new BufferedImage(img.getWidth(null), img.getHeight(null), BufferedImage.TYPE_INT_RGB);
Graphics2D g = buffImg.createGraphics();
g.drawImage(img, null, null);
return buffImg;
}
Any help greatly appreciated.
Ok, after some reading i managed to write some SwingWorker Code:
private void SingleFrameActionPerformed(java.awt.event.ActionEvent evt) {
SwingWorker<Void, BufferedImage> worker = new SwingWorker<Void, BufferedImage>() {
#Override
protected Void doInBackground() {
while (!isCancelled()) {
displayed_img = getPIC_COLOR(player);
publish(displayed_img);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
break;
}
}
return null;
}
#Override
protected void process(List mystuff) {
Iterator it = mystuff.iterator();
while (it.hasNext()) {
img_field.setIcon(new ImageIcon(displayed_img));
try {
ImageIO.write(displayed_img, "png", new File("c:\\testimg.jpg"));
} catch (IOException ex) {
Logger.getLogger(TestCAMGUI.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
#Override
protected void done() {
infoBAR.setText("FINISHED");
}
};
worker.execute();
}
Now what seriously bothers me is the fact that my img_field is not updated in swing, although ImageIO.write present in process works, and image is "redrawn" on C:\ with MAD speed ("mad speed" is highly desirable). Moreover my GUI is still frozen even though i created this swing worker thread...
btw i also tried to "sleep" this saving thread for a second, however my GUI hangs even with this additional sleep
Now some explanations about my code:
doInBackground() returns void, as i dont need to return anything, since i assume this process will run till cancelled (which is unlikely to happen, unless program is closed).
inside process() i've included try/catch block with ImageIO.write just to make sure my thread launched and works
i didnt use 'SwingUtilities.invokeLater' due to fact that browsing guides/tutorials i've read that it is better to use SwingWorker in my case
What bothers me furthermore: in process i operate on list which enlarges... i dont need that, i just need a single element and thats all. So is there a way to collect single object from doInBackground()? making list seems memory waste for me. Or maybe i should clear() the list at the end of process()? (in order to reduce memory allocated)
Any further hints are appreciated.

You must not run any kind of infinite loop of that form in a method that is called from an event (unless you explicitly intend to hang your user interface, which would be odd to say the least).
You should run your frame grab in a separate thread, and then use something like SwingUtilities.invokeLater to display it.

Related

Java/JavaFX: Poll a database for single value update, while keeping responsiveness in GUI

Together with some friends, I've tried to create a turnbased game. We have some issues regarding checking for when a user has their turn, while also keeping the GUI responsive, and also closing the thread we're using now when the game is closed. I wish to get some information on how to do this, but I'm not sure whether the problem is JavaFX-related, thread-related or both.
I've tried to search as much as I can, but simply couldn't find exactly what I'm looking for, even though I believe it is quite simple. Right now, we have a thread running a loop when you click a button to check whether or not it is your turn. When it isn't your turn, we wish to disable some user input, so I'm not sure if we actually need a thread, other than to keep the responsiveness.
I've also tried implementing a class extending Thread, but this only seemed to make the problems worse by either starting a new thread each time it wasn't the players turn, or freezing the GUI if I put the loop outside of the thread.
public void refreshButtonPressed(){
try{
refreshButton.setDisable(true);
Thread pollThread = new Thread(() -> {
System.out.println("Thread started"); //Stop being able to start more threads
int user_id = 0;
String gamePin = "xxxxxx";
while (!GameConnection.yourTurn(user_id, Context.getContext().getGamePin())){ //This method checks the database if it is your turn
try{
Thread.sleep(5000); //So we don't flood the database
}
catch (InterruptedException e){
System.out.println("Interrupted");
break;
}
//If we close the game, stop the thread/while loop.
if (TurnPolling.closedGame){
break;
}
}
playerButton.setDisable(false);
refreshButton.setDisable(false);
refreshButton.setText("Refresh");
System.out.println("Thread ended");
});
pollThread.start();
}catch (Exception e){
e.printStackTrace();
}
}
And in the controller for the gameScreen.fxml file (Not the main screen, but one loaded via login screens and the Main extending Application).
public void initialize(URL location, ResourceBundle resources) {
playerButton.setDisable(!GameConnection.yourTurn(user_id, gameTurn));
myStage.setOnCloseRequest(event -> TurnPolling.closedGame = true);
}
Right now, the TurnPolling class only has the public static boolean closedGame, so as not to keep this in the controller. The last line setting the closedGame = true actually gives me a NullPointerException, which may be because the Stage isn't initialized yet, when I do this in the initialize() method?
I would wish to enable the players buttons only when it is their turn, as well as closing the thread (if needed) when the gameScreen closes. Right now, you have to click a button to check if it is your turn, which again checks every five seconds, and it won't stop when you close the game.`
Please tell me if you need more code or clarification, this is my first big project, so I don't really know how much to put here. I know this isn't working code, but it's as much as I can do without it feeling like cluttering. Thank you for any answers!
First, it is important to remember that it is not permitted to alter JavaFX nodes in any thread other than the JavaFX application thread. So, your thread would need to move these lines:
playerButton.setDisable(false);
refreshButton.setDisable(false);
refreshButton.setText("Refresh");
into a Runnable which is passed to Platform.runLater:
Platform.runLater(() -> {
playerButton.setDisable(false);
refreshButton.setDisable(false);
refreshButton.setText("Refresh");
});
Note that changes to your TurnPolling.closedGame field in one thread may not be visible in another thread, unless it’s declared volatile. From the Java Language Specification:
For example, in the following (broken) code fragment, assume that this.done is a non-volatile boolean field:
while (!this.done)
Thread.sleep(1000);
The compiler is free to read the field this.done just once, and reuse the cached value in each execution of the loop. This would mean that the loop would never terminate, even if another thread changed the value of this.done.
Using Task and Service
JavaFX provides a cleaner solution to all this: Task and Service.
A Service creates Tasks. A Service has a bindable value property, which is always equal to the value of the most recently created Task. You can bind your button properties to the Service’s value property:
int user_id = 0;
Service<Boolean> turnPollService = new Service<Boolean>() {
#Override
protected Task<Boolean> createTask() {
return new Task<Boolean>() {
#Override
protected Boolean call()
throws InterruptedException {
updateValue(true);
String gamePin = Context.getContext().getGamePin();
while (!GameConnection.yourTurn(user_id, gamePin)) {
Thread.sleep(5000);
if (TurnPolling.closedGame){
break;
}
}
return false;
}
};
}
};
playerButton.disableProperty().bind(turnPollService.valueProperty());
refreshButton.disableProperty().bind(turnPollService.valueProperty());
refreshButton.textProperty().bind(
Bindings.when(
turnPollService.valueProperty().isEqualTo(true))
.then("Waiting for your turn\u2026")
.otherwise("Refresh"));
When the player’s turn is finished, you would call turnPollService.restart();.
Whether you use a Service, or just use Platform.runLater, you still need to make TurnPolling.closedGame thread-safe, either by making it volatile, or by enclosing all accesses to it in synchronized blocks (or Lock guards).

Swing Worker threads stop executing code unexpectedly

I am trying to create a platform for testing mutual exclusion algorithms using Swing. My intent is to display servers and messages being sent between them in the GUI. I also want to display a critical section which shows which servers are currently accessing it. I am using a threadpool that executes SwingWorker threads to deploy the servers which load a user selected mutex algorithm.
The program runs as expected until I try to implement the message displays. To display an arrow for each message I've extended SwingWorker, UIArrowThread, to add a JLabel that draws an arrow from the source server to the target. This thread waits for 1 second before removing the JLabel. This seems to be working fine when I explicitly create one or more of these messages(I've also created a test platform where I can create specific messages).
The problem comes when I try to integrate this SwingWorker thread into the program. When the algorithm is started, each server attempts to access the critical section and sends its requests to each of the other servers. This should invoke UIArrowThread but it appears that only some servers actually create the thread.
public void sendMsg(int destId, Object ... objects) {
comm.sendMsg(destId, objects);
try{
UIArrowThread a = new UIArrowThread(AlgorithmSimulatorUI.jlp,
objects[0].toString(),
comm.getMyId(),
destId);
AlgorithmSimulatorUI.threadPool.execute(a);
} catch (Exception exc){
System.err.println(exc);
}
}
Some of the servers just seem to stop executing just before instantiating UIArrowThread and ends up creating a deadlock. Any servers that do make it passed that point work normal and the GUI displays as it should be. I have testing with logging just before UIArrowThread is called and in its constructor. The threads that look like they stop executing never make the log call in the constructor. I'm pretty stumped on why this would be happening.
public class UIArrowThread extends SwingWorker<Integer, String>{
JLayeredPane jlp;
String type;
int source;
int target;
Point start;
Point end;
Point[] points;
int quadrant;
public UIArrowThread(JLayeredPane jlp, String msg_type, int source,
int target){
this.jlp = jlp;
this.type = msg_type;
this.source = source;
this.target = target;
this.points = getPoints();
this.start = points[0];
this.end = points[1];
}
#Override
protected Integer doInBackground(){
Point lblPoint = getLabelCoordinates(points);
ArrowLabel arrow = new ArrowLabel(type, 1, 2, jlp, points, quadrant);
if (quadrant < 5){
arrow.setBounds(lblPoint.x, lblPoint.y, abs(start.x - end.x),
abs(start.y - end.y));
} else if (quadrant < 7){
arrow.setBounds(lblPoint.x, lblPoint.y, 100, abs(start.y - end.y));
} else {
arrow.setBounds(lblPoint.x, lblPoint.y, abs(start.x - end.x), 100);
}
jlp.add(arrow);
String openHTML = "<html><font color='red',size=12>";
String closeHTML = "</font></html>";
arrow.setText(openHTML + type + closeHTML);
arrow.setHorizontalTextPosition(JLabel.CENTER);
arrow.setVerticalTextPosition(JLabel.CENTER);
jlp.repaint();
try{
Thread.sleep(arrow.lifespan);
} catch (Exception exc){
System.err.println(exc);
} finally {
jlp.remove(arrow);
}
jlp.repaint();
return 1;
}
I've added what I feel would be the relevant part of code for this problem. As mentioned above, if I remove the UIArrowThread, the program will run correctly.
I tried a few more approaches that still produce the same results including doing the work in process() instead of doInBackground(), and having the ArrowLabel remove itself from the GUI instead of UIArrowThread doing the removal.
Update:
I was able to get the UI working as intended but still not really sure what the original issue is. The program has a messaging queue that displays the messages from servers in a textPane so I figured I'd update the UI with the arrow labels here. It was not neccessary to alter any of the existing code for ArrowLabel or UIArrowThread.
Your fragment suggests that you are updating a Swing component, ArrowLabel, in the doInBackground() method of a SwingWorker. This violates the Swing single-thread rule. Instead, query the servers in the background, publish() interim results, and process() them on the EDT, as shown in the examples examined here. The exact formulation of "the type used for carrying out intermediate results by this SwingWorker's publish and process methods" will depend on your use case. As a concrete example, this TableSwingWorker extends SwingWorker<MyTableModel, RowData>, publishing instances of RowData used to update a TableModel

Press and hold key programmatically java

Alright, I've searched now for a whole day, but no result. Maybe someone can help it.
I'm trying to generate a key "press and hold" situation in my java program, programmatically. Let me explain the situation:
I listen constantly for an event. Consider 2 events 'A' and 'B'. If event A occurs, I want to press and hold down the keyboard key (X), and if event 'B' occurs, I want to release the key (X). Now the main thing is, this all has to be a side process, so even if A occurs, I can listen for event B.
I've tried making a separate thread, making an infinite loop inside it, pressing the key using 'Robot' in java, but it turned out to be the most inefficient way of achieving this, as it consumes more than 60% of CPU. I've tried achieving this with a state changed, but don't seem to find a way to restrict the key press, to change only when I want it to.
I would appreciate a solution without any infinite loop, if possible, as I am already using 1 to listen for event occurrences. (Suggestions are welcome)
Here is my code for the thread:
public class KeyPress implements Runnable {
public String command;
public void run() {
try {
Robot r = new Robot();
while (true) {
//System.out.println(command);
if (command.equals("up")) {
r.keyPress(KeyEvent.VK_UP);
r.delay(20);
r.keyRelease(KeyEvent.VK_UP);
} else if (command.equals("finish")) {
break;
}
}
} catch (Exception e) {
System.out.println(e);
}
}
}
The instance of thread is created as usual, in my main class.
Also, if someone can explain this - when I remove or comment out the
System.out.println(command);
statement (as you see in the code), This thread stops working. If I add this, it works. Although this problem is secondary, as it still is a non-feasible solution.
Well, after a long and tiring attempt to solve this problem, I think I might have a solution.
Firstly, I create a thread everytime event 'A' occurs, although its the same as before. When event 'B' occurs, I interrupt the thread, which makes it to exit. Since these events 'A' and 'B' occur alternatively, this works for the CPU usage problem.
Another optimization, and possibly the answer to the problem of having to write print() statement, was I made the variable command as volatile. As explained here, the compiler optimization was most likely the problem I was facing.
Here is the code after these changes:
public class KeyPress implements Runnable {
public volatile String command;
public void run() {
try {
Robot r = new Robot();
while (command.equals("up") && !Thread.currentThread().isInterrupted()) {
r.keyPress(KeyEvent.VK_UP);
r.delay(20);
}
} catch (Exception e) {
System.out.println(e);
}
}
}
I hope this helps someone, and someone can provide suggestions on how to improve it.
Observer pattern maybe a good solution.
Do not loop in the thread. Use notify and listener mode like this:
Listen to the command:
class RobotObserver implements Observer{
private Robot r = new Robot();
#Override
public void update(Observable o, Object arg) {
String command=arg.toString();
if (command.equals("up")) {
r.keyPress(KeyEvent.VK_UP);
r.delay(20);
r.keyRelease(KeyEvent.VK_UP);
} else if (command.equals("finish")) {
System.out.println(command);
}
}
}
Notify listener:
Observable observable = new Observable();
observable.addObserver(new RobotObserver());
observable.notifyObservers("up");
observable.notifyObservers("finish");
PS: class Observer and Observable are both in package java.util.

Java networking code executing before object received

I have a Java game that uses networking, and I have a client (using a Socket) fetching objects from an ObjectInputStream, running in its own thread.
From Client.java:
Object input = null;
while(true) {
input = in.readObject();
if(input != null) {
listener.gotObject(input);
}
}
This works pretty well. The object is gotten and is passed to the listener, which is a class linked to a my main GameApp class.
From the listener (NetControl.java):
public void gotObject(Object o) {
System.out.println(o);
app.gotObject(o);
}
"app" is the instance that handles all new objects received and deals with them.
From the app (GameApp.java) (edit: the non-abstract CardGameApp.java gives greater context):
public void gotObject(Object o) {
// select instance:
if(o instanceof GameList) {
GameList gameList = (GameList) o;
System.out.println("gamelist: " + gameList);
this.lobbyControl.gotGameList(gameList);
}
}
I've run this code in the debugger, one step at a time, and it works perfectly. When I run it normally though, I get a null pointer (output is as follows:)
Game ID: 0. Name: game1. Players: 1 / 1. // the object, as it is printed in Client.java
gamelist: Game ID: 0. Name: game1. Players: 1 / 1. // the object, as it is printed again in GameApp.java
Exception in thread "Thread-1" java.lang.NullPointerException
at com.lgposse.game.app.GameApp.gotObject(GameApp.java:61)
at com.lgposse.game.net.NetControl.gotObject(NetControl.java:47)
at com.lgposse.net.client.Client.run(Client.java:49)
Now, I see the object being printed twice, so I know it has been received... but I get a null pointer.
I added a sleep function in the middle of the function:
else if(o instanceof GameList) {
GameList gameList = (GameList) o;
System.out.println("gamelist: " + gameList);
try {
Thread.sleep(1000); // sleep 100 still gave null pointer
} catch (InterruptedException e) {}
this.lobbyControl.gotGameList(gameList);
}
And setting it to sleep for a while, it all finally worked.
Any idea why I need to sleep the thread like this? Is there something I should do differently? I'm not sure why I was able to print the object while it was still considered null.
Edit: added some more context.
It looks like lobbyControl is null, not gameList. If gameList were null, the top of the stack would be the gotGameList() method, not gotObject().
If sleeping helps the problem, then you must be manipulating the lobbyControl member without proper concurrency safeguards. An ObjectInputStream won't return an object until it's been fully read from the stream, so your problem has nothing to do with not having completely read the object.
Update: I can't follow all the code, but it appears that a reference to the object being constructed is leaked to a thread (the client in the NetControl), which is started before the constructor completes. If that is the case, that's very, very bad. You should never allow a partially constructed object to become visible to another thread.
Well, I'll start off by saying that the code snippets posted seem to help illustrate the issue, but i don't think the full picture is painted. I'd ask for a bit more code, to help get a full context.
That being said, I'd offer the following guidance:
Don't lean on java's built in object serialization. It's nice and
easy to use, but can be very unstable and error prone at runtime.
I'd suggest a custom object serialization and deserialization
scheme.
Depending on the scope of the game you're making, NIO may be a
netter choice. If you stick with regular IO, then make sure you
have a rock solid Thread Manager in place to properly handle the
threads dealing with the socket IO.
..without more code, that's the most I can offer.
Just to improve my comment...When i need to wait for one or more threads to finish, i like to use java.util.concurrent.CountDownLatch. Its very simple:
//game class
public class DummyGame
{
CountDownLatch signal;
public DummyGame( CountDownLatch signal)
{
this.signal = signal;
}
public void run()
{
doLogic();
signal.countDown();
}
}
//game controller class
public void run()
{
while (! gameOver)
{
CountDownLatch signal = new CountDownLatch(1); //wait one thread to finish
new thread(newGame(signal)).start();
//wait for game run() to finish
signal.await();
updateInterface();
}
}
That's just an idea, hope it helps.

Java - Multiple Objects controlled by Timer (for a Game)

I'm currently working on a small game that consisted of a few targets that pop up at random for various amounts of time. The actual game will get it's I/O from a circuit board since the targets are physical.
My problem is that currently I have a java.util.Timer that fire's off every 2 seconds. Once it is triggered a random target will be displayed (which works fine so far). The problem is that I want to display the targets for a random number of seconds between 1-5 whilst the timer is still running and setting off other targets.
I get no errors and the targets display but never disappear. I guess it's some sort of Thread issue and that maybe since I'm using this.* the Target objects are just somehow getting lost in the nether! After searching around the questions here I have come up with this:
public class Target implements Runnable(){
...
public void displayFor(int seconds){
this.display();
Executors.newSingleThreadScheduledExecutor().schedule(this,time,
TimeUnit.SECONDS);
this.setDisplayed(false);
}
#Override
public void run() {
this.destroy();
}
}
Basically the initial game timer (that sets of the Targets display) calls the displayFor(2) method which runs the targets run method after the time passed. The Targets still won't disappear though.
I have tried a number of different ways of doing this like the displayFor() setting off another java.util.Timer and I also had a go at using the Quartz library (which to be honest seemed like overkill anyway) and still can't get it to work. Since there are no error messages I'm really stuck with this one.
I've haven't included a lot of the code because I don't think it's that relevant but if you guys need more information to help just let me know :)
I managed to get it working. Here's the correct code for anyone in a similar situation.
public class Target{
private Timer timer;
...
public void displayFor(int seconds) {
// send the output
BoardInterface.SetDigitalChannel(this.getId());
// calculate the delay
long time = seconds * 1000;
// create a new timer and schedule the new task
timer = new Timer();
timer.schedule(new TargetTimer(this), time);
this.setDisplayed(true);
}
}
class TargetTimer extends TimerTask{
Target target;
public TargetTimer(Target t){
this.target = t;
}
#Override
public void run() {
target.destroy();
}
}
Not sure if this is a good way of doing it but it works. If you notice anything that could be improved please let me know. Thanks guys!
Perhaps you should tell us what the display method does.
Are you un-displaying the target in the destroy/destructor code?
I'd recommend, instead of void display():
public void setDisplayed(boolean display){
if(display) {
///... do appropriate circuit output to turn on target
} else {
/// ... do appropriate circuit output to turn off target
}
}
and of course
public void run(){
setDisplayed(false);
destroy();
}

Categories

Resources