I have the task to stream an IP camera's video stream (RTP/RTSP in h264) via J2EE application server to a browser. For this I am using GStreamer 1.21.3 (latest dev release) with the gstreamer-java library on top. We are aiming towards a Websocket solution as the traditional HLS introduces significant latency.
After having figured out what to do with the gst-launch executable on the commandline, I ended up with this code (for the moment):
/*
* Configuration for RTSP over TCP to WebSocket:
* 1. rtspsrc to ip camera
* 2. rtph264depay ! h246parse to extract the h264 content
* 3. mp4mux to create fragmented MP4
* 4. appsink to grab the frames and use them in Websocket server
*/
final String gstPipeline = String.format("rtspsrc onvif-mode=true protocols=tcp user-id=%s user-pw=%s location=%s latency=200"
+ " ! rtph264depay ! h264parse"
+ " ! mp4mux streamable=true fragment-duration=5000"
+ " ! appsink name=sink", USERNAME, PASSWORD, uri);
final Pipeline pipeline = initGStreamerPipeline(gstPipeline);
// Add listener to consume the incoming data
final AppSink sink = (AppSink) pipeline.getElementByName("sink");
sink.setCaps(Caps.anyCaps());
sink.set("emit-signals", true);
sink.set("max-buffers", 50);
sink.connect((NEW_SAMPLE) appsink -> {
final Sample sample = appsink.pullSample();
if (sample == null)
{
return FlowReturn.OK;
}
final Buffer buffer = sample.getBuffer();
try
{
final ByteBuffer buf = buffer.map(false);
LOGGER.debug("Unicast HTTP/TCP message received: {}", new String(Hex.encodeHex(buf, true)));
if (session != null)
{
try
{
buf.flip();
session.getRemote().sendBytes(buf);
}
catch (final Exception e)
{
LOGGER.error("Failed to send data via WebSocket", e);
}
}
}
finally
{
buffer.unmap();
}
return FlowReturn.OK;
});
sink.connect((AppSink.EOS) s -> LOGGER.info("Appsink is EOS"));
sink.connect((AppSink.NEW_PREROLL) s -> {
LOGGER.info("Appsink NEW_PREROLL");
return FlowReturn.OK;
});
LOGGER.info("Connecting to {}", uri);
/**
* Start the pipeline. Attach a bus listener to call Gst.quit on EOS or error.
*/
pipeline.getBus().connect((Bus.ERROR) ((source, code, message) -> {
LOGGER.info(message);
Gst.quit();
}));
pipeline.getBus().connect((Bus.EOS) (source) -> Gst.quit());
pipeline.play();
/**
* Wait until Gst.quit() called.
*/
LOGGER.info("Starting to consume media stream...");
Gst.main();
pipeline.stop();
server.stop();
Now I seem to be stuck here, because the AppSink at the end of the pipeline never gets its new_sample signal triggered. The complete example works like a charme when I replace the appsink with a filesink. I have noticed that there are some other threads (like this one) with similar problems which normally boil down to "you forgot to set emit-signals=true". Any ideas why my appsink gets no data?
Update:
It appears that the problem is the URL I am passing to the pipeline string. It has two query parameters: http://192.168.xx.xx:544/streaming?video=0&meta=1. If I remove the second parameter (and the ambersand along with it), the pipeline works. Unfortunately I found no docs how to escape URLs in the correct way so GStreamer can read it. Can anyone share such documentation?
Update 2:
It starts getting weired now: It looks like the name of the URL parameter is the problem. I have started to replace it with some dummy argument and it works. So the ambersand is not the problem. Then I used VLC media player to consume the stream with the &meta=1 in place which also worked. Is it possible that the string "meta" is treated special in gstreamer?
I am using Smack 4.1.7 on the android application and also on the server side.
I would like to transfer Files from the phone to the server so i use the following code on the phone:
FileTransferNegotiator.IBB_ONLY = true;
FileTransferManager fileTransferManager = FileTransferManager.getInstanceFor(connection);
OutgoingFileTransfer transfer = fileTransferManager.createOutgoingFileTransfer("fulljid");
transfer.sendFile(file);
and on the server side:
FileTransferNegotiator.IBB_ONLY = true;
fileTransferManager = FileTransferManager.getInstanceFor(xmppConnection);
fileTransferManager.addFileTransferListener(new FileTransferListener() {
public void fileTransferRequest(FileTransferRequest request) {
IncomingFileTransfer fileTransfer = fileTransferRequest.accept();
File file = new File("destinationpath");
fileTransfer.receiveFile(file);
}
});
It just works if I have the line IBB_ONLY = true; in my code, but I have no idea why - can anyone tell me?
The next question is: it takes more than two minutes to upload about 200kB from the phone to the server. The same file needs not even 5 seconds to be uploaded to dropbox for example. Is this because of the IBB_ONLY or is there another issue?
Thanks for your help!
Update:
Is there a better way to do it?
I am working on implementing SSE in a Web Application on Java Stack using Servlets. I have facing 2 key issues currently. Let me first place my code both for the Web page and the Servlet followed by the issue I am facing.
Web Page Code:
<script type="text/javascript">
function registerSSE() {
var source = new EventSource("http://www.sample.com/BootStrap/NotificationServlet");
source.addEventListener('StartProductionRun', function(e) {
// Get the data and identify the instrument Name/Id
var dataReceived = e.data;
document.getElementById(dataReceived + "_button").disabled = true;
}, false);
}
function StartProduction(instrument) {
var dataString = 'instrumentName='+ instrument;
// call ajax to submit the form and start the production run
$.ajax({
type: 'POST',
url: 'http://localhost:8080/BootStrap/ProductionRunServlet',
data: dataString,
success: function() {
$('#Status').html("<div id='message'></div>");
$('#message').html("<h4 aling=\"centre\">Prudction Run for Instrument " + instrument + " initiated.</h4>")
.hide()
.fadeIn(5000);
}
});
}
</script>
Servlet Code :
public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/event-stream;charset=UTF-8");
response.setHeader("Cache-Control", "no-cache");
PrintWriter out = response.getWriter();
NotificationService notificationService = null;
while (true) {
notificationService = NotificationService.getInstance();
if (notificationService.getNotificationCount() > 0 ) {
String notificationValue = notificationService.getNotification(0);
String[] keyValue = notificationValue.split(":");
out.print("event:" + keyValue[0] + "\n");
out.print("data: " + keyValue[1] + "\n");
out.print("retry:" + 1000 + "\n\n");
out.flush();
}
else {
out.print(": time stream \n");
out.print("retry:" + 1000 + "\n\n");
out.flush();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Now the issues:
The Web Page will be viewed by multiple users at the same time. And I want that the data to be pushed to all the users who viewing that page. Currently when I am running locally in my machine, even if I open Chrome and Firefox, I don't get the notification in both the Browsers. It comes only in one.
Also, If I leave the browser running for some time, I find that even if the servlet is pushing out data based on certain events. I don't get the notification on the Browser.
I need to make sure that:
The notification gets pushed to all the clients who are viewing that particular page irrespective of what they are doing on the page or the page is just used for viewing the information.
Looking forward to all the help I can get to make this working. Also, would be interested to know if there are other alternative which I can use.
Your servlet code is working perfectly, although i haven't worked so much on it(once did a jsp project). What i think is, you have missed something in javascript?
I think there should be a timer/thread/loop in javascript too, to get all pushed data continuously. i.e,
setInterval(
function(){
// code which needs to run every 5sec
},5000);
I hope this will help out a bit.
You should check if EventSource is available in that browser before using it. Maybe one of the browsers has not support for it.
I'm evaluating Node.js for possible replacement of my current push functionality on a Java Web App. I wrote a simple long polling server that works like an intermediary between the client and the Java back-end. The client makes a request to subscribe, and then the Java server can notify subscribed clients by calling Node.js. It seems to be working fine so far, but I got the following message which points to a memory leak:
(node) warning: possible EventEmitter memory leak detected. 11 listeners added.
Use emitter.setMaxListeners() to increase limit.
Trace
at EventEmitter.addListener (events.js:168:15)
at EventEmitter.once (events.js:189:8)
at route (C:\Users\Juan Pablo\pushserver.js:42:12)
at Server.onRequest (C:\Users\Juan Pablo\pushserver.js:32:3)
at Server.EventEmitter.emit (events.js:91:17)
at HTTPParser.parser.onIncoming (http.js:1793:12)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:111:23
)
at Socket.socket.ondata (http.js:1690:22)
at TCP.onread (net.js:402:27)
I have a line of code that logs the existing listeners whenever a notify event is emitted. I've had it running for a while and it shows that there is only one listener per subscribed client (as should be), but this line wasn't on the code when I got the warning message. The code was exactly the same except for that line tough.
This is the push server's code (it's a bit rudimentary since I'm still learning Node.js):
var http = require('http');
var url = require("url");
var qs = require("querystring");
var events = require('events');
var util = require('util');
var emitter = new events.EventEmitter;
function onRequest(request, response)
{
var pathname = url.parse(request.url).pathname;
console.log("Request for " + pathname + " received.");
request.setEncoding("utf8");
if (request.method == 'POST')
{
var postData = "";
request.addListener("data", function(postDataChunk)
{
postData += postDataChunk;
console.log("Received POST data chunk '"+ postDataChunk + "'.");
});
request.addListener("end", function()
{
route(pathname, response, postData);
});
}
else if (request.method=='GET')
{
var urlParts = url.parse(request.url, true);
route(pathname, response, urlParts.query);
}
}
function route(pathname, response, data)
{
switch (pathname)
{
case "/subscription":
emitter.once("event:notify", function(ids)
{
response.writeHead(200, {"Content-Type": "text/html", "Access-Control-Allow-Origin": "*"});
response.write(JSON.stringify(ids));
response.end();
});
break;
case "/notification":
//show how many listeners exist
console.log(util.inspect(emitter.listeners('event:notify'));
emitter.emit("event:notify", data.ids);
response.writeHead(200, {"Content-Type": "text/html", "Access-Control-Allow-Origin": "*"});
response.write(JSON.stringify(true));
response.end();
break;
default:
console.log("No request handler found for " + pathname);
response.writeHead(404, {"Content-Type": "text/plain", "Access-Control-Allow-Origin": "*"});
response.write("404 - Not found");
response.end();
break;
}
}
http.createServer(onRequest).listen(8888);
console.log('Server running at http://127.0.0.1:8888/');
I was under the impression that using emitter.once would automatically remove the event listener once it was used, so I don't know how 11 listeners could've been added if there was only one client connected. I'm thinking that perhaps if the client disconnects while waiting for a notification then the associated connection resources are not disposed.
I'm wondering whether I have to manually handle disconnections and if there is actually a leak in there somewhere. Any advice is welcome. Thanks.
If anyone is interested, the above code does leak. The leak occurs when a client disconnects before a notification is sent. To fix this, it is necessary to remove the event listener when a client disconnects abruptly, such as:
case "/subscription":
var notify = function(ids)
{
response.writeHead(200, {"Content-Type": "text/html", "Access-Control-Allow-Origin": "*"});
response.write(JSON.stringify(ids));
response.end();
}
emitter.once("event:notify", notify);
//event will be removed when connection is closed
request.on("close", function()
{
emitter.removeListener("event:notify", notify);
});
break;
This is solved at last with "timeout" attribute of jQuery AJAX (and JSONP). See my own answer !
Please see the updated part, I have tried with applet too. And will not hesitate to accept your answer if you can give a solution with applet implementation.
I am working with a Java based web application. My requirement is to check whether a particular port (say 1935) is open or blocked at client's end. I have implemented a "jsonp" (why 'jsonp' ? i found that 'http' request through AJAX cannot work for corssdomain for browsers 'same origin policy') AJAX call to one of my server containing particular port. And if the server returns xhr.status == 200 the port is open. Here is a drawback that I can't make the execution-flow wait (synchronous) until the call completes. Here is the JavaScript function I am using.
Any alternative solution (must be a client-sided thing must be parallel with my application, please dont suggest python/php/other languages) is also welcome. Thanks for your time.
function checkURL() {
var url = "http://10.0.5.255:1935/contextname" ;
var isAccessible = false;
$.ajax({
url: url,
type: "get",
cache: false,
dataType: 'jsonp',
crossDomain : true,
asynchronous : false,
jsonpCallback: 'deadCode',
complete : function(xhr, responseText, thrownError) {
if(xhr.status == "200") {
isAccessible = true;
alert("Request complete, isAccessible==> " + isAccessible); // this alert does not come when port is blocked
}
}
});
alert("returning isAccessible=> "+ isAccessible); //this alert comes 2 times before and after the AJAX call when port is open
return isAccessible;
}
function deadCode() {
alert("Inside Deadcode"); // this does not execute in any cases
}
---------------------------------------------------------UPDATE----------------------------------------------------------------
I have tried with Java Applet (thanks to Y Martin's suggestion). This is working fine in appletviewer. But when I add the applet in HTML page, it is giving vulnerable results. Vulnerable in the sense, when I change the tab or resize the browser, the value of portAvailable is being altered in the printed message.
Applet Code :
import java.applet.Applet;
import java.awt.Graphics;
import java.net.InetSocketAddress;
import java.net.Socket;
public class ConnectionTestApplet extends Applet {
private static boolean portAvailable;
public void start() {
int delay = 1000; // 1 s
try {
Socket socket = new Socket();
/*****This is my tomcat5.5 which running on port 1935*************/
/***I can view it with url--> http://101.220.25.76:1935/**********/
socket.connect(new InetSocketAddress("101.220.25.76", 1935), delay);
portAvailable = socket.isConnected();
socket.close();
System.out.println("init() giving---> " + portAvailable);
}
catch (Exception e) {
portAvailable = false;
System.out.println("init() giving---> " + portAvailable);
System.out.println("Threw error---> " + e.getMessage());
}
}
public void paint(Graphics g) {
System.out.println("Connection possible---> " + portAvailable);
String msg = "Connection possible---> " + portAvailable;
g.drawString(msg, 10, 30);
}
}
And this is my HTML page (I am hosting it on same computer with a different Tomcat 6 which runs on port 9090. I can view this page with url ---> http://101.220.25.76:9090/test/):
<html>
<body>
<applet code="ConnectionTestApplet" width=300 height=50>
</applet>
</body>
</html>
And how I am doing the port 1935 blocking and openning ?
I have created firewall rule for both inbound and outbound for port 1935.
I check the port 1935 open/blocked scenario by disabling/enabling both rules.
This is my S.S.C.C.E. Now please help me :)
Gotcha !!! I have solved my problem with JSONP and jQuery AJAX call. I discovered the timeout attribute of jQuery AJAX and my code executed fluently when the port was blocked or opened. Here is the solution for future visitors. Thanks to all answerers for contribution.
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<script type="text/javascript" src="jquery-1.7.2-min.js"></script>
</head>
<body>
<script type"text/javascript">
var isAccessible = null;
function checkConnection() {
var url = "http://101.212.33.60:1935/test/hello.html" ;
$.ajax({
url: url,
type: "get",
cache: false,
dataType: 'jsonp', // it is for supporting crossdomain
crossDomain : true,
asynchronous : false,
jsonpCallback: 'deadCode',
timeout : 1500, // set a timeout in milliseconds
complete : function(xhr, responseText, thrownError) {
if(xhr.status == "200") {
isAccessible = true;
success(); // yes response came, esecute success()
}
else {
isAccessible = false;
failure(); // this will be executed after the request gets timed out due to blockage of ports/connections/IPs
}
}
});
}
$(document).ready( function() {
checkConnection(); // here I invoke the checking function
});
</script>
</body>
</html>
I don't think you understand the use cases for JSONP and it's not possible to test open ports with it. http://en.wikipedia.org/wiki/JSONP
If you want a client side solution it could be possible with websockets, but this is only available on new browsers like chrome or ff. Otherwise request a server side script which does the ping. For example - with a curl script: curl and ping - how to check whether a website is either up or down?
Here is a Java code as an Applet to test server/port connectivity:
import java.io.*;
import java.net.*;
public class ConnectionTestApplet extends Applet {
public void start() {
boolean portAvailable = false;
int delay = 1000; // 1 s
try {
Socket socket = new Socket();
socket.connect(new InetSocketAddress("server.domain.com", 1935), delay);
portAvailable = socket.isConnected();
socket.close();
} catch (UnknownHostException uhe) {
uhe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
System.out.println("Connection possible: " + portAvailable);
}
}
You still have to get the information out of the applet to do something else with that result. The easiest way is to redirect the browser thanks to getAppletContext().showDocument(url)
Instead of an applet a flash component may be used. Using the Socket class available in ActionCcript one can open a tcp connection from flash to a port on a server to check if its open. But based on the flash player version a policy file needs to be placed on the server to which the socket is opened.
Check this out:
http://blog.andlabs.org/2010/12/port-scanning-with-html5-and-js-recon.html
With JS-Recon, you can do port scanning with javascript. You can simply point it to your local IP address. I believe it works by making a web sockets/cors connection to an arbitrary desintation ip/socket and measuring the timeouts. It is not a perfect approach, but this may be the limit of javascript ultimately.
If you can do it in a java applet/flash application, that may be better ultimately as they have lower-level access.
You cannot do this in JavaScript because it doesn't have true socket support, with JavaScript you can only test for the presence of HTTP socket. You could use Java (JavaScript is not Java) and write a proper Java Applet to do it.
You should also read this Q&A How to Ping in java
Try using isReachable
In JavaScript, you have to work-around the asynchronous issue. Here is a proposal:
The HTML page displays an animated image as a progress bar
You invoke the checkURL
After either receiving the callback or a defined timeout, you change display for an error message or do on with the job to do
Based on the following document with the use of XMLHttpRequest, here is a code example for checkURL:
var myrequest = new ajaxRequest();
var isAccessible = false;
myrequest._timeout = setTimeout(function() {
myrequest.abort();
displayErrorMessage();
},
1000
) //end setTimeout
myrequest.onreadystatechange = function() {
if (myrequest.readyState == 4) { //if request has completed
if (myrequest.status == 200) {
isAccessible = false;
goOnWithTheJob();
} else {
displayErrorMessage();
}
}
myrequest.open("GET", url, true);
myrequest.send(null); //send GET request
// do nothing - wait for either timeout or readystate callback
This code lets 1 second to get the 200 response from a HTTP GET on a basic resource.
In your local test, you get an immediate answer because the system answers connection reset if the port is closed but a firewall just does not answer.
Even if the open method may be used synchronously, I recommend the use of a timer because the code is likely to wait for TCP timeouts and retries (3 x 1 minute ?) as a firewall usually just drops packets on closed ports and may reject ICMP packets, preventing you to test availability thanks to ping. And I imagine such a long wait is not expected for such a check.
I am occasional frontend/javascript/jQuery guy, so this may not be 100% professional, but it is good enough and it solved my similar problem:
ping_desktop_app = $.get({
url: "http://127.0.0.1:#{desktop_app_port}",
dataType: 'jsonp',
})
$(#).parent().find(".click-me-to-use-desktop-app").click ->
if ping_desktop_app.status == 200
$.get({
url: "http://127.0.0.1:#{desktop_app_port}/some_command/123123",
dataType: 'jsonp',
})
else
alert("Please run your desktop app and refresh browser")
I could not check whether port is open (desktop app is running) on server side because views are cached, so I needed to check the localhost/port right before user click in browser
Edits translating to JS are welcome