I'm making a player that can play an MPEG-TS stream and display all its videos at once (for monitoring purposes) in one frame using Xuggler for JAVA.
My problem is getting to determine what programs this stream holds (tv programs) and what are its streams...
for example : audio stream 1 and video stream 3 belong to program "BBC".
Now I already got it working for a .ts file by using MediaInfo http://mediaarea.net/en/MediaInfo/ like so :
MediaInfo.exe -LogFile="log.txt" "some .ts file" .... which logs a file like this :
Menu #2
ID : 1001 (0x3E9)
Menu ID : 1202 (0x4B2)
Duration : 13mn 33s
List : 2001 (0x7D1) (MPEG Video) / 3002 (0xBBA) (MPEG Audio, English)
Language : / English
Service name : NBN
Service provider : NILESAT
Service type : digital television
UTC 2006-03-28 00:00:00 : en:NBN / en:Nilesat / / / 99:00:00 / Running
and then I parsed the file in java
but I need to make this work for a live stream and when I give MediaInfo a URL instead of a file it gives this error :
Libcurl library not found
I also tried vlc commands but turns out it doesn't have this option and only available in gui (show codec information)...
the player is already working and I got an extractor too... just need this media info to work... any ideas?
EDIT : I found out FFprobe which is bundled with FFmpeg http://www.ffmpeg.org/ can do the task
but for some reason I can't read anything from the input stream.
Here's what the output looks like:
Input #0, mpegts, from 'D:\record ts\PBR_REC_20140426094852_484.ts':
Duration: N/A, start: 6164.538011, bitrate: N/A
Program 1201
Metadata:
service_name : Arabica TV
service_provider: Nilesat
Stream #0:10[0x7db]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv42
0p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 2348 kb/s, 25 fps, 25 tbr, 90k tbn, 50
tbc
Stream #0:4[0xbcf]: Audio: mp2, 48000 Hz, stereo, s16p, 384 kb/s
Program 1202
I tried this in JAVA:
try {
Process process ;
Scanner sc;
ProcessBuilder processBuilder = new ProcessBuilder("C:\\Users\\vlatkozelka\\Desktop\\ffmpeg-20140623-git-ca35037-win64-static\\bin\\ffprobe.exe","-i",filename);
process=processBuilder.start();
sc=new Scanner(process.getInputStream());
process=processBuilder.start();
while(sc.hasNext()){
System.out.println(sc.nextLine());
}
} catch (IOException ex) {
Logger.getLogger(ChannelDivider.class.getName()).log(Level.SEVERE, null, ex);
}
but the sc.hasNext() just hangs like there is no input
then I tried writing to a file with cmd by using > but it gave me a blank file
however trying both methods with FFprobe -h (help command) does give output which is very much confusing me, I see output in cmd but cant read it...
I just solved this, and hope someone might make use of it:
it turns out that FFprobe was writing to stderr, not stdout,
so instead of:
getInputStream()
I used:
getErrorStream()
Now all I have to do is parse that :)
Related
I have a grayscale .mkv video, which i want to open with OpenCV in Java, but i get the following errors:
With return new VideoCapture(path, Videoio.CAP_FFMPEG);
Errors:
[ERROR:0#0.004] global /build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1108) open Could not find decoder for codec_id=61
[ERROR:0#0.004] global /build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture
With return new VideoCapture(path, Videoio.CAP_DSHOW); No errors, but
video.isOpened() is false
With return new VideoCapture(path);
Errors:
[ERROR:0#0.005] global /build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1108) open Could not find decoder for codec_id=61
[ERROR:0#0.005] global /build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture
[ WARN:0#0.122] global C:\build\master_winpack-bindings-win64-vc14-static\opencv\modules\videoio\src\cap_msmf.cpp (923) CvCapture_MSMF::initStream Failed to set mediaType (stream 0, (480x360 # 1) MFVideoFormat_RGB24(codec not found)
I have installed OpenCV and added it as a dependency using this video.
I have also tried adding ...\opencv\build\bin\opencv_videoio_ffmpeg455_64.dll to the native libraries, and also tried using this: System.load("path\\to\\opencv\\build\\bin\\opencv_videoio_ffmpeg455_64.dll");.
Full code:
public class Test {
static {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
System.load("path\\to\\opencv\\build\\bin\\opencv_videoio_ffmpeg455_64.dll");
}
public static void main(String[] args) {
List<Mat> frames = getVideoFrames(openVideoFile(args[0]));
System.out.println(frames.size());
}
}
//... different class
public static VideoCapture openVideoFile(String path) {
return new VideoCapture(path);
}
public static List<Mat> getVideoFrames(VideoCapture video) {
List<Mat> frames = new ArrayList<>();
Mat frame = new Mat();
if (video.isOpened()) {
while (video.read(frame)) {
frames.add(frame);
}
video.release();
}
return frames;
}
ffprobe result:
Metadata:
MAJOR_BRAND : qt
MINOR_VERSION : 512
COMPATIBLE_BRANDS: qt
ENCODER : Lavf56.40.101
Duration: 00:01:05.83, start: 0.000000, bitrate: 1511 kb/s
Stream #0:0(eng): Video: png (MPNG / 0x474E504D), rgb24(pc), 480x360 [SAR 1:1 DAR 4:3], 6 fps, 6 tbr, 1k tbn (default)
Metadata:
LANGUAGE : eng
HANDLER_NAME : DataHandler
ENCODER : Lavc56.60.100 png
DURATION : 00:01:05.834000000
The error message is indicating that OpenCV's FFmpeg plugin is not built with MPNG codec support. So, you are essentially SOL to get this task done only with OpenCV (you can request OpenCV to support the codec, but it won't be a quick adaption even if you succeed to convince their devs to do so). Here are a couple things I could think of as a non-Java/non-OpenCV person (I typically deal with Python/FFmpeg):
1a) If you have a control of the upstream of your data, change the video codec from MPNG to one with OpenCV support.
1b) Transcode the MKV file to re-encode the video stream with a supported codec within your program
Create a thread and call ffmpeg from Java as a subprocess (I think ProcessBuilder is the class you are interested in) and load the video data via stdout pipe. FFmpeg can be called in the following manner:
ffmpeg -i <video_path> -f rawvideo -pix_fmt gray -an -
The stdout pipe will receive 480x360 bytes per frame. Read as many frames as you need at a time. If you need to limit the frames, you need to do this in seconds using -ss, -t, and/or -to options.
I'm assuming this video is grayscale as you mentioned (ffprobe is indicating the video is saved in RGB format). If you need to get RGB, use -pix_fmt rgb24 and the video frame data.
Once you have the image data in memory, there should be an OpenCV function to create an image object from in-memory data.
Trying to communicate with a BLE device (smart lamp).
I use the following dependency:
<dependency>
<groupId>com.github.hypfvieh</groupId>
<artifactId>bluez-dbus</artifactId>
<version>0.1.3</version>
</dependency>
Very interesting library, by far the best I have found so far for BLE, in term of code quality, dependency management, and clarity...
The problem is, I have a working gatttool command like this, running on a Raspberry Pi 4:
gatttool --device=C4:AC:05:42:73:A4 -t random --char-write-req -a 0x1f -n a001037F
... which set the brightness of the lamp to 100%. Note the value of the address (ie. "-a 0x1f"), which corresponds to the attribute "char value handle" in gattool "characteristics":
handle: 0x001e, char properties: 0x28, char value handle: **0x001f**, uuid: **44092842-0567-11e6-b862-0002a5d5c51b**
I try to make the same using bluez-dbus in java. My implementation seems correct, but the lamp doesn't respond. I have the following trace with dbus-monitor :
method call time=1600276508.729104 sender=:1.184 -> destination=org.bluez serial=210 path=/org/bluez/hci0/dev_C4_AC_05_42_73_A4/service001d/**char001e**; interface=org.bluez.GattCharacteristic1; member=WriteValue
array of bytes [
0a 01 03 7f
]
array [
]
method return time=1600276508.776261 sender=:1.5 -> destination=:1.184 serial=6589 reply_serial=210
It looks everything is fine except bluez-dbus pick up the value 0x001e (aka. the "handle" in gatttool characteristics), to drive the lamp, where it should have been 0x001f ("char value handle" in gatttool).
Do you know if this is a bad usage of the library, an error on the device, or what ?
Here is a little excerpt of the code, if you need more you can look here: https://github.com/sebpiller/luke-roberts-lamp-f
BluetoothDevice lampF = manager.getDevices(true)
.stream()
.filter(e -> Objects.equals(e.getAddress(), config.getMac()))
.findFirst()
.get();
....
String uuid = config.getCustomControlService().getUuid();
BluetoothGattService customControlService = Objects.requireNonNull(lampF.getGattServiceByUuid(uuid));
LOG.info("found GATT custom control service {} at UUID {}", customControlService, uuid);
....
String externalApiUuid = config.getCustomControlService().getUserExternalApiEndpoint().getUuid();
externalApi = Objects.requireNonNull(customControlService.getGattCharacteristicByUuid(externalApiUuid));
...
private void sendCommandToExternalApi(LukeRoberts.LampF.Command command, Byte... parameters) {
reconnectIfNeeded();
try {
externalApi.writeValue(/*reversed*/ command.toByteArray(parameters), Collections.emptyMap());
} catch (DBusException e) {
throw new IllegalStateException("unable to change brightness: " + e, e);
}
}
Thanks for your time !
EDIT:
I am an idiotic-dyslexic. 0x0a is not the same as 0xa0.
Sometimes I'd like to crush my head on the wall....
Thanks for your help :)
gattool is one of the eight tools that have been deprecated by BlueZ.
To debug this I would advise using bluetoothctl to workout what the correct paths are for the connected device. A session might look like this:
pi#raspberrypi:~ $ bluetoothctl
[bluetooth]# connect C4:AC:05:42:73:A4
[my lamp]# menu gatt
[my lamp]# select-attribute 44092842-0567-11e6-b862-0002a5d5c51b
[my lamp:/service0032/char0036]# write 0xa0 0x01 0x03 0x7F
Attempting to write /org/bluez/hci0/dev_C4_AC_05_42_73_A4/service0032/char0036
On the command line, to show you all the paths can be done with generic D-Bus tools:
pi#raspberrypi:~ $ busctl tree org.bluez
Once you have the paths then you can do it from the command line with D-Bus.
pi#raspberrypi:~ $ busctl call org.bluez /org/bluez/hci0/dev_DE_82_35_E7_43_BE org.bluez.Device1 Connect
pi#raspberrypi:~ $ busctl call org.bluez /org/bluez/hci0/dev_DE_82_35_E7_43_BE/service0032/char0036 org.bluez.GattCharacteristic1 WriteValue aya{sv} 4 0xa0 0x01 0x03 0x7f 0
Hopefully with the knowledge from these experiments you can better understand what is going on with the Java application.
I'm using the JavaCV FFmpegFrameRecorder class to encode Android's camera preview frames into a video.
The goal would be to replicate the result of the following command line:
ffmpeg -i input.mp4 -metadata:s:v:0 rotate="90" output.mp4
I modified the startUnsafe() method as follows, but it failed to generate the desired output:
if ((video_st = avformat_new_stream(oc, video_codec)) != null) {
video_c = video_st.codec();
video_c.codec_id(oformat.video_codec());
video_c.codec_type(AVMEDIA_TYPE_VIDEO);
...
AVDictionary avDictionary = new AVDictionary(null);
av_dict_set(avDictionary, "rotate", "90", 0);
video_st.metadata(avDictionaty);
...
}
...
avformat_write_header(oc, (PointerPointer) null);
This still encodes the video correctly, but the added metadata never appears on ffprobe. If it helps, the video encoding is h264.
By the way, here's the ffprobe output:
ffprobe version 2.3.3 Copyright (c) 2007-2014 the FFmpeg developers
built on Jan 22 2015 18:22:57 with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.3.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-libvorbis --enable-libvpx --enable-libass --enable-ffplay --enable-libfdk-aac --enable-libopus --enable-libquvi --enable-libx265
libavutil 52. 92.100 / 52. 92.100
libavcodec 55. 69.100 / 55. 69.100
libavformat 55. 48.100 / 55. 48.100
libavdevice 55. 13.102 / 55. 13.102
libavfilter 4. 11.100 / 4. 11.100
libavresample 1. 3. 0 / 1. 3. 0
libswscale 2. 6.100 / 2. 6.100
libswresample 0. 19.100 / 0. 19.100
libpostproc 52. 3.100 / 52. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'abcd.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf56.15.102
Duration: 00:00:19.48, start: 0.023220, bitrate: 572 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, 573 kb/s, 5.71 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 64 kb/s (default)
Metadata:
handler_name : SoundHandler
Any suggestions on why is it failing? Thanks.
It seems this question generated a lot of interest so I'm adding some more info. Following this GitHub issue Samuel from JavaCV committed some changes to allow easier access to metadata settings.
Setting metadata can be achieved through the following code snippet:
AVDictionary metadata = new AVDictionary(null);
for (Entry<String, String> e : videoMetadata.entrySet()) {
av_dict_set(metadata, e.getKey(), e.getValue(), 0);
}
video_st.metadata(metadata);
You can enable it right now by doing mvn install -Pffmpeg or wait until the next JavacV release, which should be 0.12.
PS: As you can see, this is pretty similar to what I presented in my question, so I'm not sure why it didn't work in the first place.
The FFmpegFrameRecorder class you are using uses an AVFormatContext class. Around line 2579 you can see from the method signatures, that the AVFormatContext class uses native code to implement both
public native AVDictionary metadata() method
public native AVFormatContext metadata(AVDictionary metadata) method.
The answer to the link you provided says they used the metadata property of the AVFormatContext directly - something like the first method I think. But Line 649 of FFmpegFrameRecorder uses the second method - I suspect. i.e.:
AVDictionary metadata = new AVDictionary(null);
... code to fill up dictionary ...
...
avformat_write_header(oc.metadata(metadata), options);
Unfortunately I can't try this at the moment, but I wonder if you could do something like this:
AVDictionary metadata = co.metadata();
... code to fill up dictionary ...
//I would assume at this point that oc has the metadata so
avformat_write_header(oc, (PointerPointer) null);
//if not then maybe
// avformat_write_header(oc.metadata(metadata), options);
The signature shows it is public, so I don't see why you can't get the metadata dictionary from the AVFormatContext directly. I'm not sure how the avformat_write_header method works, so I suggested two things above.
Note: I have not used this library before. I tried, unsuccessfully to use Xuggler in the past for some basic encoding.
Short: I'd like to know the name of this format!
I would like to know if this is a special common format or just a simple self-generated config file:
scenes : {
Scene : {
class : Scene
sources : {
Game Capture : {
render : 1
class : GraphicsCapture
data : {
window : "[duke3d]: Duke Nukem 3D Atomic Edition 1.4.3 STABLE"
windowClass : SDL_app
executable : duke3d.exe
stretchImage : 0
alphaBlend : 0
ignoreAspect : 0
captureMouse : 1
invertMouse : 0
safeHook : 0
useHotkey : 0
hotkey : 123
gamma : 100
}
cx : 1920
cy : 1080
}
}
}
}
My background is, that I would like to read multiple files like this one above. And I don't want to implement a whole new parser for this. That's why I want to fall back on java libraries which have already implemented this feature. But without being aware of such code formats, it's quite difficult to search for this libraries.
// additional info
This is a config file or a "scene file" for Open Broadcaster Software.
Filename extension is .xconfig
This appears to be a config file or a "scene file" for Open Broadcaster Software.
When used with OBS it has a extension of .xconfig
Hope this helps.
-Yang
I got some feedback from the main developer of this files.
As i thought, this is not a know format - just a simple config file.
solved!
I have a program called FFprobe which probes media (files/live streams ... ) and outputs result in different formats , for example :
ffprobe.exe -i test.ts -print_format xml -show_programs
gives this output :
<?xml version="1.0" encoding="UTF-8"?>
<ffprobe>
<programs>
<program program_id="1201" program_num="1201" nb_streams="2" pmt_pid="1000" pcr_pid="2011" start_pts="45394227044" start_time="45394.227044">
<tag key="service_name" value="Arabica TV"/>
<tag key="service_provider" value="Nilesat"/>
<streams>
<stream index="10" codec_name="mpeg2video" codec_long_name="MPEG-2 video" profile="Main" codec_type="video" codec_time_base="1/50" codec_tag_string="[2][0][0][0]" codec_tag="0x0002" width="720" height="576" has_b_frames="1" sample_aspect_ratio="16:15" display_aspect_ratio="4:3" pix_fmt="yuv420p" level="8" timecode="08:28:54:09" id="0x7db" r_frame_rate="25/1" avg_frame_rate="25/1" time_base="1/90000" start_pts="4085542516" start_time="45394.916844" max_bit_rate="2348000">
<disposition default="0" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0"/>
</stream>
<stream index="4" codec_name="mp2" codec_long_name="MP2 (MPEG audio layer 2)" codec_type="audio" codec_time_base="1/48000" codec_tag_string="[0][0][0][0]" codec_tag="0x0000" sample_fmt="s16p" sample_rate="48000" channels="2" channel_layout="stereo" bits_per_sample="0" id="0xbcf" r_frame_rate="0/0" avg_frame_rate="0/0" time_base="1/90000" start_pts="4085480434" start_time="45394.227044" bit_rate="384000">
<disposition default="0" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0"/>
</stream>
</streams>
</program>
... more programs
to retrieve this info in java i used ProcessBuilder and a scanner , and then id write to a file once the result is ok ... but it wasnt :
Process proc = new ProcessBuilder("ffprobe.exe","-i", ... same arguments );
Scanner sc = new Scanner (proc.getInputStream()) // im 100% sure its not errorStream
while(sc.hasNext()){
System.out.println(sc.nextLine());
}
the app just hangs with no output , i know its hanging cuz the process is still running and scanner has next , but , i don't know why it would do that ?If i execute the same in cmd i would get good result and ofc i can write to file with ">"
Ive tried it w/o the -print_format option , which gives the info in a plain text on the errorstream(i know its error cuz i was able to write with 2> not >> ) , and i was able to read the error stream in java , but its not meant for parsing cuz very very un-organized .
Input #0, mpegts, from 'PBR_REC_20140426094852_486.ts':
Duration: 00:13:34.30, start: 7791.344722, bitrate: 42154 kb/s
Program 1201
Metadata:
service_name : Arabica TV
service_provider: Nilesat
Stream #0:19[0x7db]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 2348 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:3[0xbcf]: Audio: mp2, 48000 Hz, stereo, s16p, 384 kb/s
Program 1202
Metadata:
service_name : NBN
service_provider: NILESAT
Stream #0:10[0x7d1]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 2600 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:11[0xbba](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 125 kb/s
Program 1203
Metadata:
service_name : Heya TV
service_provider: NILESAT
Stream #0:5[0x7d2]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 2600 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:6[0xbbc](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, mono, s16p, 125 kb/s
Program 1204 ... more programs
now it might seem organized and "parse-able" and i actually made a parser that worked to some point , but sometimes it doesnt stick to this structure and ruins the whole parsing, its why i need a xml/json ...
The -print_format im sure outputs to outputstream.
Im not asking for help on how to use FFprobe as thats not this place's purpose , Im asking why am i not being able to get the output stream from java while it is definitely outputting if i execute in windows .
I also tried apache commons-exec as i know processbuilder can be a pain ,it did execute perfectly and showed in System.in (black for intput and red for error), but getting the stream with apache is something i couldn't understand , i tried this example
The xml parser i already taken care of , simply put i just need to execute that first command from java and read the output , but for some reason its not working .
I generally use variations of this utility method:
public static void runProcess(ProcessBuilder pb) throws IOException {
pb.redirectErrorStream(true);
Process p = pb.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}
Even if you don't want to print the error stream you need to consume it otherwise it will block the process. In which case you would need to consume it in a separate thread and only print the input stream from the main thread.