Develop Reference

GStreamer java: AppSink not receiving new_sample signal

I have the task to stream an IP camera's video stream (RTP/RTSP in h264) via J2EE application server to a browser. For this I am using GStreamer 1.21.3 (latest dev release) with the gstreamer-java library on top. We are aiming towards a Websocket solution as the traditional HLS introduces significant latency.
After having figured out what to do with the gst-launch executable on the commandline, I ended up with this code (for the moment):
/*
* Configuration for RTSP over TCP to WebSocket:
* 1. rtspsrc to ip camera
* 2. rtph264depay ! h246parse to extract the h264 content
* 3. mp4mux to create fragmented MP4
* 4. appsink to grab the frames and use them in Websocket server
*/
final String gstPipeline = String.format("rtspsrc onvif-mode=true protocols=tcp user-id=%s user-pw=%s location=%s latency=200"
+ " ! rtph264depay ! h264parse"
+ " ! mp4mux streamable=true fragment-duration=5000"
+ " ! appsink name=sink", USERNAME, PASSWORD, uri);
final Pipeline pipeline = initGStreamerPipeline(gstPipeline);
// Add listener to consume the incoming data
final AppSink sink = (AppSink) pipeline.getElementByName("sink");
sink.setCaps(Caps.anyCaps());
sink.set("emit-signals", true);
sink.set("max-buffers", 50);
sink.connect((NEW_SAMPLE) appsink -> {
final Sample sample = appsink.pullSample();
if (sample == null)
{
return FlowReturn.OK;
}
final Buffer buffer = sample.getBuffer();
try
{
final ByteBuffer buf = buffer.map(false);
LOGGER.debug("Unicast HTTP/TCP message received: {}", new String(Hex.encodeHex(buf, true)));
if (session != null)
{
try
{
buf.flip();
session.getRemote().sendBytes(buf);
}
catch (final Exception e)
{
LOGGER.error("Failed to send data via WebSocket", e);
}
}
}
finally
{
buffer.unmap();
}
return FlowReturn.OK;
});
sink.connect((AppSink.EOS) s -> LOGGER.info("Appsink is EOS"));
sink.connect((AppSink.NEW_PREROLL) s -> {
LOGGER.info("Appsink NEW_PREROLL");
return FlowReturn.OK;
});
LOGGER.info("Connecting to {}", uri);
/**
* Start the pipeline. Attach a bus listener to call Gst.quit on EOS or error.
*/
pipeline.getBus().connect((Bus.ERROR) ((source, code, message) -> {
LOGGER.info(message);
Gst.quit();
}));
pipeline.getBus().connect((Bus.EOS) (source) -> Gst.quit());
pipeline.play();
/**
* Wait until Gst.quit() called.
*/
LOGGER.info("Starting to consume media stream...");
Gst.main();
pipeline.stop();
server.stop();
Now I seem to be stuck here, because the AppSink at the end of the pipeline never gets its new_sample signal triggered. The complete example works like a charme when I replace the appsink with a filesink. I have noticed that there are some other threads (like this one) with similar problems which normally boil down to "you forgot to set emit-signals=true". Any ideas why my appsink gets no data?
Update:
It appears that the problem is the URL I am passing to the pipeline string. It has two query parameters: http://192.168.xx.xx:544/streaming?video=0&meta=1. If I remove the second parameter (and the ambersand along with it), the pipeline works. Unfortunately I found no docs how to escape URLs in the correct way so GStreamer can read it. Can anyone share such documentation?
Update 2:
It starts getting weired now: It looks like the name of the URL parameter is the problem. I have started to replace it with some dummy argument and it works. So the ambersand is not the problem. Then I used VLC media player to consume the stream with the &meta=1 in place which also worked. Is it possible that the string "meta" is treated special in gstreamer?

Blocking read function in c in a client/server application

I'm currently developing a client/server architecture between a tablet (client) and a MAC/PC (server). I am doing on both side some real-time rendering and I need communication between the two.
The problem is that I need to do some operation on the string I get from my client (which is basically a rotation matrix). This string is therefore to be at most 16 float numbers that I previously transform into a coma-separated-value string.
Therefore what I should get from my client is something like:
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
Server-side, I do some processing of that string to get back my rotation matrix as a float array of 16 elements. The problem is that sometimes I get more than just 16 elements from the client on the server side at once. I for instance get
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
So that when I try to split it, I go above the 16 element limits which is not good at all for me.
My question is: is there a way to prevent the server and/or the client to read/send more than one complete matrix at a time? Since I'm using a tablet and some real-time rendering I would like to be able to save as much processing power as possible.
Here is the code that I'm using (just snippets as files are quite big)
Client:
if (connected == true && matrixupdated == true && this.hasMatrixChanged()){
try {
this.inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
this.outToServer= new DataOutputStream(clientSocket.getOutputStream());
this.sentence = this.getStringFromMatrix();
outToServer.writeBytes(sentence + '\n');
this.hasServerProcessed = false ;
System.arraycopy(matrix, 0, previousMatrix, 0, 16); //I check whether the matrix changes enough for me to send it to the server
}catch (Exception e) {
Log.e("ClientActivity", "S: Error", e);
}
this.matrixupdated = false ;
Server :
while( (read_size = recv(sock , client_message , 2000 , 0)) > 0 )
{
smatrix = client_message ; //smatrix is a true c++ string
pthread_mutex_lock(&mymutex);
pthread_cond_wait(&mycondition, &mymutex); // prevent real-time rendering to try and use the matrix at the same time as this function
std::stringstream ss(smatrix);
while(std::getline(ss, tok, ',')) {
matrix[i] = ::atof(tok.c_str());
i++ ;
}
i = 0 ;
pthread_mutex_unlock(&mymutex);
}

Working as designed. TCP is a byte stream protocol. There are no message boundaries. If you want messages you have to implement them yourself.

Odd InetAddress.isReachable() issue

My work is developing software for network capable cameras for retail enviroments. One of the peices of software my team is developing is a webserver that retrieves various reports generated in HTML by the camera itself (which has its own embedded webserver) and stored on the camera. Our software will then GET these reports from the camera and store it on a central webserver.
While we are fine plugging in the IPs of the cameras into our software, I am developing a simple Java class that will query the network and locate all cameras on the network.
The problem though is that while it runs just fine on my PC, and my coworker's PC, when we attempt to run it on the actual webserver PC that will host our software... it runs, but says every IP in the subnet is offline / unreachable EXCEPT for the gateway IP.
For example, if I run it from my PC or my coworkers PC when plugged into the closed LAN, I get the following active IPs found along with a flag telling me if its a camera or not.
(gateway is 192.168.0.1, subnet mask is 255.255.255.0, which means full range of 256 devices to be looked for)
IP:/192.168.0.1 Active:true Camera:false
IP:/192.168.0.100 Active:true Camera:true <- this is camera 1
IP:/192.168.0.101 Active:true Camera:true <- this is camera 2
IP:/192.168.0.103 Active:true Camera:false <- my PC
IP:/192.168.0.104 Active:true Camera:false <- this is our webserver
But for some reason, when running the same program from the webserver PC, using the same JRE, I only get the following found
IP:/192.168.0.1 Active:true Camera:false
Now my code, instead of enumerating through each IP in order on the main Thread, instead creates a seperate Thread for each IP to be checked and runs them concurrently (else it would take little over 21 minutes to enumerate through the entire IP range at a timeout of 5000ms / IP). The main Thread then re-runs these IP scan threads every 15 seconds over and over.
I have checked that all the threads are running to completion on all the PCs, no exceptions are being thrown. Even verified that none of the threads are getting stuck. Each Thread takes about 5001 to 5050ms from start to complete, and those Threads that have an active IP finish sooner (>5000ms), so I know that its correctly waiting the full 5000ms in the ipAddr.isReachable(5000) method.
Me and my coworker are stumped at this point while it seems to reach those active IPs fine when run on our PCs, yet getting no response from the webserver PC???
We have ruled out firewall issues, admin access issues, etc.. The only difference is that our webserver is Embedded Win XP, and our PCs are Windows 7.
This has us stumped. Any ideas why?
Below is the code that is running each IP Thread:
public void CheckIP() {
new Thread() {
#Override
public void run() {
try {
isActive = ipAddr.isReachable(5000);
if (isActive) {
if (!isCamera) {
isCamera = new IpHttpManager().GetResponse(ipAddr.toString());
}
} else {
isCamera = false;
}
} catch (Exception e) {
e.printStackTrace();
}
}
}.start();
}
EDIT: Here is the code that builds each IP to check after determining the range based on gateway and subnet...
for(int i=subMin; i<=subMax; i++) {
byte[] ip = new byte[] {(byte)oct[0],(byte)oct[1],(byte)oct[2],(byte)i};
try {
scanners[subCount] = new IpScan(InetAddress.getByAddress(ip));
subCount++;
} catch (UnknownHostException e) {
e.printStackTrace();
}}

Thanks everyone, but I never did figure out or pinpoint why this oddity was happening. Everything I checked for was not the cause, so this question can be closed.
In any case, I ended up working around it completely. Instead of using InetAddress, I just went native and built my own ICMP ping class instead, via JNA, invoking Windows libraries IPHLPAPI.DLL and WSOCK32.DLL. Here is what I used...
public interface InetAddr extends StdCallLibrary {
InetAddr INSTANCE = (InetAddr)
Native.loadLibrary("wsock32.dll", InetAddr.class);
ULONG inet_addr(String cp); //in_addr creator. Creates the in_addr C struct used below
}
public interface IcmpEcho extends StdCallLibrary {
IcmpEcho INSTANCE = (IcmpEcho)
Native.loadLibrary("iphlpapi.dll", IcmpEcho.class);
int IcmpSendEcho(
HANDLE IcmpHandle, //Handle to the ICMP
ULONG DestinationAddress, //Destination address, in the form of an in_addr C Struct defaulted to ULONG
Pointer RequestData, //Pointer to the buffer where my Message to be sent is
short RequestSize, //size of the above buffer. sizeof(Message)
byte[] RequestOptions, //OPTIONAL!! Can set this to NULL
Pointer ReplyBuffer, //Pointer to the buffer where the replied echo is written to
int ReplySize, //size of the above buffer. Normally its set to the sizeof(ICMP_ECHO_REPLY), but arbitrarily set it to 256 bytes
int Timeout); //time, as int, for timeout
HANDLE IcmpCreateFile(); //win32 ICMP Handle creator
boolean IcmpCloseHandle(HANDLE IcmpHandle); //win32 ICMP Handle destroyer
}
And then using those to create the following method...
public void SendReply(String ipAddress) {
final IcmpEcho icmpecho = IcmpEcho.INSTANCE;
final InetAddr inetAddr = InetAddr.INSTANCE;
HANDLE icmpHandle = icmpecho.IcmpCreateFile();
byte[] message = new String("thisIsMyMessage!".toCharArray()).getBytes();
Memory messageData = new Memory(32); //In C/C++ this would be: void *messageData = (void*) malloc(message.length);
messageData.write(0, message, 0, message.length); //but ignored the length and set it to 32 bytes instead for now
Pointer requestData = messageData;
Pointer replyBuffer = new Memory(256);
replyBuffer.clear(256);
// HERE IS THE NATIVE CALL!!
reply = icmpecho.IcmpSendEcho(icmpHandle,
inetAddr.inet_addr(ipAddress),
requestData,
(short) 32,
null,
replyBuffer,
256,
timeout);
// NATIVE CALL DONE, CHECK REPLY!!
icmpecho.IcmpCloseHandle(icmpHandle);
}
public boolean IsReachable () {
return (reply > 0);
}

My guess is that your iteration logic to determine the different ip address is based upon different configuration hence your pc's fetches all addresses but your webserver doesn't.
Try adding debug in the logic where you build up the list of ip adresses to check.

Can not read response from server

I have written a Java Program and the program connects to a database on my server, to find records, write records, update and delete. for some reason finding records works, but most of the time when i try to save or write a record it gives an error message saying:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 9,787 milliseconds ago. The last packet sent successfully to the server was 8,183 milliseconds ago.
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2552)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
... 46 more
Can anyone explain why this is happening?
Usuually this gives me the error message when trying to add a record, after i had the software running for more than about half a minute. seems to loose connection or something. when i run the program and quickly write a new record, it works

I was having the same sort of issue. I referred many post and comments but the thing worked for me was changing some parameters of the my.cnf file. Hope it will help you also ....
Set following parameters in my.cnf [mysqld] section
interactive_timeout=180 # "No.of sec. a server waits for activity on interactive connection before closing it"
wait_timeout=180 # "No. of sec. a server waits for an activity on a connection before closing it"
max_connect_errors=9999 # "More than this number of interrupted connections from a host this host will be blocked from further connections"
skip-name-resolve # "Don't resolved host names. All host names are IP's"

Sometimes this problem comes due to size of system RAM.May be you are inserting the data using buffer through RAM. To get out of this problem.
Set the Auto commit disable before inserting the data.
insert some amount of data appropriate to your System RAM (not the
whole).
Commit the query.
Do the steps 2 and 3 again until the whole insertion will not be
done.
You can understand this by the following code.
public static void main(string args[])
{
Connection con = null;
Statement stm = null;
int i;
float ratio;
ratio=1.0f;
try
{
Class.forName("com.mysql.jdbc.Driver");
// Connecting to the database
con = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3306/demo",
"ashish", "impetus");
File f = new File("filler"); // taking the random text data from the file
// filler.txt and inserting that string
// in filler field of the relations
RandomAccessFile r = new RandomAccessFile(f,"r");
Random randomGenerator = new Random();
String strLine=new String();
int r1,flag=0;
stm = con.createStatement();
con.setAutoCommit(false) ;
int k=0;
i=0;
long sum2=0;
while(k%50000==0)
{
final long start = currentTimeMillis();
final StringBuilder builder =
new StringBuilder("INSERT INTO accounts
(aid, bid,abalance,filler) VALUES ");
while(i!=naccounts*ratio*scale )
{
int j=i+1;
for(int l=0;l<40;l++)
{
strLine+=((char)r.read());
r.skipBytes(0);
}
r1=randomGenerator.nextInt(1500);
if(strLine.equals(""))
{
flag=1;
}
if(flag!=1)
{
strLine=strLine.replaceAll("\\s","");
strLine=strLine.replaceAll("\\t","");
}
flag=0;
if (i%50000!=0)
{
builder.append(",");
}
builder.append(format("(%s, %s, %s, '%s')", j,
i/naccounts+1, 0, strLine));
strLine="";
r.seek(r1);
i++;
if(i%50000==0||i>=naccounts*ratio*scale)
{
final String query = builder.toString();
final PreparedStatement statement1 = con.prepareStatement(query);
statement1.execute();
con.commit();
final long stop= currentTimeMillis();
sum2=sum2+(stop-start);
statement1.close();
}
if(i%50000==0||i>=naccounts*ratio*scale)
{
break;
}
}
k=k+50000;
if(k>naccounts*ratio*scale)
{
break;
}
}
System.out.println(i+" rows inserted accounts table ");
System.out.println("time taken = "+sum2+" milliseconds");
}
catch (SQLException e)
{
System.out.println("Connection Failed! Check output console");
e.printStackTrace();
return;
}
catch (Exception e)
{
e.printStackTrace();
}
}

Did you follow/read this tutorial :
Connectivity with MYSQL
You have a part for your exception which can be useful for you.
I will quote something about your specific exception, just try that :
If you get a SQLException: Connection refused or Connection timed out
or a MySQL specific CommunicationsException: Communications link
failure, then it means that the DB isn't reachable at all. This can
have one or more of the following causes:
IP address or hostname in JDBC URL is wrong. Hostname in JDBC URL is
not recognized by local DNS server. Port number is missing or wrong in
JDBC URL. DB server is down. DB server doesn't accept TCP/IP
connections. DB server has run out of connections. Something in
between Java and DB is blocking connections, e.g. a firewall or proxy.
To solve the one or the other, follow the following advices:
Verify and test them with ping. Refresh DNS or use IP address in JDBC
URL instead. Verify it based on my.cnf of MySQL DB. Start the DB.
Verify if mysqld is started without the --skip-networking option.
Restart the DB and fix your code accordingly that it closes
connections in finally. Disable firewall and/or configure
firewall/proxy to allow/forward the port.

Serial Communication between Java RXTX and Arduino

I'm trying to communicate between my PC (Windows 7 using Netbeans and RXTX) with an Arduino Pro, using the serial port. The Arduino is actually connected to the PC using an FTDI cable.
The code is based on the Java SimpleRead.Java found here.
Currently the Arduino simply prints out a string when it starts up. My Java program should print the number of bytes that have been read and then print out the contents. The Java program works, sort of...
If the string is long (>10 bytes or so) the output will get broken up.
So if on the Arduino I print
Serial.println("123456789123456789"); //20 bytes including '\r' and '\n'
The output of my Java program may look something like:
Number of Bytes: 15
1234567891234
Number of Bytes: 5
56789
or
Number of Bytes: 12
1234567891
Number of Bytes: 8
23456789
I'm thinking it's a timing problem, because when I manually go through the code using the debugger, the result string is always what it should be: one 20 byte string.
I've been messing with various things but I haven't been able to fix the problem.
Here is the part of the code that is giving me problems:
static int baudrate = 9600,
dataBits = SerialPort.DATABITS_8,
stopBits = SerialPort.STOPBITS_1,
parity = SerialPort.PARITY_NONE;
byte[] readBuffer = new byte[128];
...
...
public void serialEvent(SerialPortEvent event)
{
if (event.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
try {
if (input.available() > 0) {
//Read the InputStream and return the number of bytes read
numBytes = input.read(readBuffer);
String result = new String(readBuffer,0,numBytes);
System.out.println("Number of Bytes: " + numBytes);
System.out.println(result);
}
} catch (IOException e) {
System.out.println("Data Available Exception");
}
}

Serial data is just a stream of data. Depending on when you read it and the buffering that is happening, only part of the data may be available when you read it.
Since you are using line oriented data, what you will want to do is buffer the data until you see the line terminator and only then process the data.

I haven't used Java RXTX, but I've played with Arduino and Processing and it's pretty easy to read/write values from Arduino.
Here is a read sample that comes with Processing(File > Examples > Libraries > Serial > SimpleRead)
/**
* Simple Read
*
* Read data from the serial port and change the color of a rectangle
* when a switch connected to a Wiring or Arduino board is pressed and released.
* This example works with the Wiring / Arduino program that follows below.
*/
import processing.serial.*;
Serial myPort; // Create object from Serial class
int val; // Data received from the serial port
void setup()
{
size(200, 200);
// I know that the first port in the serial list on my mac
// is always my FTDI adaptor, so I open Serial.list()[0].
// On Windows machines, this generally opens COM1.
// Open whatever port is the one you're using.
String portName = Serial.list()[0];
myPort = new Serial(this, portName, 9600);
}
void draw()
{
if ( myPort.available() > 0) { // If data is available,
val = myPort.read(); // read it and store it in val
}
background(255); // Set background to white
if (val == 0) { // If the serial value is 0,
fill(0); // set fill to black
}
else { // If the serial value is not 0,
fill(204); // set fill to light gray
}
rect(50, 50, 100, 100);
}
/*
// Wiring / Arduino Code
// Code for sensing a switch status and writing the value to the serial port.
int switchPin = 4; // Switch connected to pin 4
void setup() {
pinMode(switchPin, INPUT); // Set pin 0 as an input
Serial.begin(9600); // Start serial communication at 9600 bps
}
void loop() {
if (digitalRead(switchPin) == HIGH) { // If switch is ON,
Serial.print(1, BYTE); // send 1 to Processing
} else { // If the switch is not ON,
Serial.print(0, BYTE); // send 0 to Processing
}
delay(100); // Wait 100 milliseconds
}
*/
As far as I remember, the baud thingy you setup in Arduino when you instantiate Serial is pretty important. If you use 9600 to send for example, you should use the same number to listen.
Also it's pretty important to send your information as BYTE, otherwise you'll have stuff like \r or \n in the way.
Shorter version, try:
Serial.println(123456789123456789,BYTE);
The simpler the better.

I think you need to use event driven design patterns to solve this problem. I highly recommend you to visit: http://www.whatisarduino.org/bin/Tutorials/Java+Serial+API+and+Arduino