I want to get some advice. I have to implement java server which will stream video from user to other clients. On client side will be using WebRTC. Can you give me some ideas how to implement this? Thank you)
You should use a media server that has WebRTC capabilities to do that. Don't try to write one on your own.
Here are a few open source alternatives: Janus, Jitsi, Kurento, mediasoup
If you plan on using an open source one, then I suggest you also understand the realities of these options - some got acquired, which changed a bit how they operate and how solid a solution they may be for you: https://bloggeek.me/twilio-acquires-kurento/
Janus video room demo is doing that. Although Janus is not written in Java, and I would say it is less than rock solid, it works quite well for this purpose.
We could not make Kurento working (last year). It seemed having odd requirements of a specific Linux distribution and version, and even once it was working fine, it just stopped working for no reason a month after.
Kurento does offer a lot more than Janus, OTOH you don't need those features. Using simple WebRTC and Janus makes this use case possible.
There is no need to write any Java server. Clients can send the video stream right from a HTML5 webpage, and viewers can view it right in HTML5. Janus examples show the actual code.
Related
Since I got no answer and not much feedback on this question: Android Flutter Analyze Audio Waveform
and found nothing online about what I'm looking for, I'll simply ask a broader question, since a comment on that answer told me to use native code and use a platform channel to connect it to flutter but when I asked some clarifications I got nothing.
So my question is If I can do operations in Java (which has been around since a much longer time, and thus has a way bigger documentation), and then use the outcome in Flutter.
More precisely, could I do these things in Java and Flutter:
1) Analyse Audio waveform and find peak points in specific frequencies, and use the timestamp to display them in flutter;
Edit 1:
What Are Peak Points?
This Is the waveform of different frequencies ranges (The Orange one is bass (80-255Hz)), and the Points circled in black are Peak Points. I should analyze the audio specter of a song and Find the peak points in certain frequencies. Then When I Find the Peaks I need to save the timestamps, for example, 16 seconds in and so on.
2) Edit 2:
I need to Edit some photos in a video, like a video collage, for which each frame of a 30 or 60fps video is an image.
3) Edit 3:
I need to add basic frame specific effects to the video, for example a blur that will change frame to frame, or a glare.
4) Adding Music to that video and save it to an mp4 or avi or any format.
5) Edit 4: Most Important thing, I don't want to this all in real time, but more like an After Effect like Render process, in which all the frames are rendered together. The Only thing that would be nice is a sort of progress bar telling the user that the Render is at frame, for example, 200 of 300, but I don't want to display any of the frames or the video, just to render it in background and then save it to an mp4 video that can be viewed after.
As You can see it's a difficult process to do in a language to which you hardly find a tutorial on how to play music due to its early state. But Uis and some other things in flutter are way easier to do and Flutter is also Multi-Platform. So I prefer to stick to Flutter language.
Edit 5:
I took a look at Qt and JUCE, and found out that Qt seems a valid alternative but it seems for what understood more like a "closed" system, I mean, for example I looked the multimedia library but for what I've understood, you can do basic stuff, for example play a video, but not collage frames and save it. (Don't know if I explained myself well). JUCE On the other side, looks better but it seems more for PC audio VST than for mobile applications including video rendering. And another thing is that these two are not free and open source like Flutter is.
Then There is Kivy, which could and could not be the best, because it is a Python port for Mobile Devices and I have a lot of experience with Python And I think it's one of the easier language to learn, but on the other side, it hasn't got that much UI power. and as you mentioned there could be problem using libraries on Android.
You stated I could use C++ or Java With Flutter, but with C++ you told that it's a difficult process. So My question turned out to be Could I write the process in java with a Normal Android Application And Then in some way use the functions in a Flutter App?
Edit 6:
I found a possivle alternative:
Kha (http://kha.tech/). But again found nothing on how to use it with Flutter. Could it be a good Idea?
I'm asking more of a confirmation on if I could use Java or any other language to do what I need in a Flutter Application. And If yes if it's complicated or not that much. (I'm a beginner sorta). But Some tutorial or links to kickstart the code would be helpful aswell!
Flutter at this time is great for building UIs but as you've mentioned, it doesn't have a lot of power or compatibility with libraries yet. A good part of the reason for that is that it doesn't have easy integration with c++, but I won't get into that now.
What you're asking is most likely possible but it's not going to be simple at all to do. First, it sounds like you're wanting to pull particular frames from a video, and display them - that's going to be an additional complication. And don't forget that on a mobile device you have somewhat limited processing power - things will have to be very asynchronous which can actually cause problems for flutter unless you're careful.
As to your points:
This is a very general ask. I'd advise looking up android audio processing libraries. I'm almost sure it's possible, but SO questions are not meant for asking advise on which framework to use. Try https://softwarerecs.stackexchange.com/.
Once again, fairly general and a bit unclear about what you're asking... Try sofwarerecs. I assume you're wanting to take several frames and make them into a video?
Some of those effects (i.e. zoom) you could definitely do with flutter using a Transform. But that would just be while playing in flutter rather then adding to the video files themselves. To do that, you'll have to use the video library in android/java code.
Once again, the video library should do this.
This should also be part of the video library.
I do know of one audio/video library off the top of my head called Processing that may do what you need, but not for sure. It does have an android sdk though. OpenCV would be another but only for video/image processing and I haven't used it directly with Java so I'm not sure how easy it is to use.
For how you'd actually go about implementing this along with flutter... you're going to need to use Platform Channels. I mentioned them in the comment to your other answer but figured you could look that up yourself. The documentation does do a much better job of explaining how that works and how to set it up than I can. But the TLDR is that essentially, what they allow you to do is to send serialized data from native code (java/kotlin/swift etc) to flutter code (dart) and vice-versa, which gets translated into similar data structures in the target language. You can set up various 'channels' upon which the data flows, and within those channels set up 'methods' which get called at either end, or simply send events back and forth.
The complication I mentioned at the beginning is that sending images back and forth across the channels between flutter and dart isn't all that optimal. You most likely won't get a smooth 24/30/60fps of images being sent from java to dart, and it might slow down the rest of the flutter ui significantly. So what you'll want to use for the actual viewport is instead a Texture, which simply displays data from the android side. You'll have to figure out how to write to a texture from android yourself, but there's lots of information available for that. Controls, the visualization of the audio, etc can be done directly in flutter with data that is retrieved from native.
What you'll have is essentially a remote control written in dart/flutter, which sends various commands to a audio/video processing library & wrapper code in Java.
If all that sounds complicated, that's because it is. And as much as flutter is very nice to build UIs in, I have doubts as to whether it's going to be worth the extra complications if you're only targeting android.
Not really related to the answer but rather some friendly advice:
There is one other thing I'll mention - I don't know your level of proficiency with programming and with different languages, but video/audio processing and such are generally not done in java but rather in actual native code (i.e. c/c++). As such, there are actually two levels of abstraction you're going to have to be dealing with here (to some degree as it will probably be abstracted somewhat or a lot depending on the library you're using) - c/c++ to java and java to dart.
You may want to cut out the middlemen and work more directly with native - in that case I'd recommend at least taking a look at Qt or JUCE as they may be more suitable than flutter for your particular use case. There's also Kivy (uses python) which may work well as there's a ton of image/video/audio processing libraries for Python somehow... although they may not all work on android and still have the c++ => python translation to some degree. You'll have to look into licensing etc though - Qt has a broad enough OS licence for most android apps, but JUCE you'd have to pay for unless you're doing open source. I'd have to recommend Qt slightly more than the others as it actually has native decoding of video frames etc, although you'd probably want to incorporate OpenCV or something for the more complicated effects you are talking about. But it would probably be on the same level of complicated as simply writing in java code, but with a slightly different UI style & easier integration with c++ libraries.
I need to set up live streaming from a number of web-cameras to the internet (in browsers), and the streams should be visible only to particular users. I.e. user A logs in to my system with his or her login/password, goes to the video stream page, and sees the stream from a particular cam, and other users cannot see that video, even if they know the url to that stream.
I've looked at a number of solutions so far, but some of them are obsolete, most of them are for image processing, recognition and the like, and some are just a bit too cumbersome, like Red5, for example.
Is there a relatively simple solution for that, that would just allow me to get a videostream from a particular cam connected to my computer?
Thanks in advance.
If you are using Linux I have had success with V4L4J (Video 4 Linux 4 Java) which is small and really quite cool. You would need to do quite a bit of work to get streaming working for a low bandwidth connection but if it is over LAN playing back an MJPEG stream over a TCP socket is easy beans :-)
http://code.google.com/p/v4l4j/
Good luck.
Have a look at Java Media Framework to communicate with your cam.
How can I get video and audio streams from web cameras with Java (in a cross-platform way)?
For example, we have a computer with 3-4 USB web cameras; we want to get their streams and make them visible in the user interface. How can we perform such a thing? I need code for a simple app which would find ALL cameras on the computer and let us select camera to wach. Can any one, please share such one?
With java media framework JMF or FMJ
Here is an example with JVM
I see your comment about the solution needing to be "pure" java, but this is really "nearly" impossible. You will need to use some sort of native library for accessing the devices, with that being stated; I suggest that you look at Xuggler as your solution. Xuggler supports Windows, Mac, and Linux and they also offer example code for handling audio and video. Lastly the library is FREE.
Webcam capture example: http://xuggle.googlecode.com/svn/trunk/java/xuggle-xuggler/src/com/xuggle/xuggler/demos/DisplayWebcamVideo.java
While Xuggle does not have an example of microphone capture, you can find samples on the mailing list or you could use code written by the packet-mulitbroadcaster project found here: https://code.google.com/p/packet-multibroadcaster/
You can try my solution. It uses OpenCV to capture the image and jetty server to transmit it via WebSocket. The example has three classes and it will be easy to understand.
Now I can send only video, but I'm working on audio aswell.
I'd use flex. It can be relatively easily integrated with java backend.
UPD:
Pure Java needed
Then you should consider JavaFX solutions. I'm not a big expert in javafx, I've only written some basic test applications, but I'm sure it's a modern way of solving described problem with pure java.
I'm thinking about making a physical controller (device?) with knobs, buttons, and LEDs. I'd like to interact with it using Java (respond to the knobs, light up LEDs, etc). The reason I mention Java is two-fold: first, I know Java well1. Second, I've written the rest of the program I need to interface with in Java (though there are ways to talk to the Java program from another language).
I would like the device to connect via USB and be (computer-)platform independent.
I haven't the slightest idea of where to start, except to start reading the Arduino website. Is this my best/only option? Is there something better suited for communicating with Java?
Note: I know that Arduino has something to do with Java (not sure what), but it seems like code must be written in a subset of C.
How would I get moving on this topic?
1 - No laughter, please.
The Arduino development environment is written in Java.
But the standard language you write a program for the Arduino platform is effectively C++.
The Arduino platform is based on an Atmel AVR chip. There is at least one Java VM for AVR chips. There are other languages available for the AVR such as Forth and BASIC (although I could only find commercial versions, so I'll if you want to find them, search for "AVR BASIC").
The Arduino uses a virtual COM port to communicate between the host computer and it. A virtual COM port emulates an old style serial line but is done with USB. You can use the Java communication API to then have a Java program running on the host computer communicate with your physical device.
For some encoders and buttons, you probably want to implement a USB HID device. If you're going to produce more than a couple of them, you'll want to do a custom board. Check out V-USB, an open-source library for making USB HID devices using Atmel microcontrollers. They have a bunch of examples of projects that use this library.
You could probably make this look like a HID joystick, using the encoders to produce X/Y axis information and having the buttons act like buttons. They you could use standard Java gaming APIs to read the joystick values.
Can you expand on your need for a custom device? It seems to me that designing hardware has a pretty high barrier to entry and that most applications I can think of would be better resolved by repurposing an existing piece of game controller hardware. If you really need new hardware, then i suggest you start by googling 'USB development kit' or
'USB development board' which will get you links like this, this and this.
As for working with USB hardware from Java, I've played around with the JUSB library a bit and it looks promising, but appears to be limited to Linux only because the Windows version of the native library half of the library hasn't been written. Alternatives can by found by googling 'HID java'.
Ok, computer-platform independant? What platforms are you targetting? That would depend on the driver and support for that said platform, what does the usb device do? Is it a mass storage device...You may have to look around and see if you can find a device driver that can talk to the device...
Hope this helps,
Best regards,
Tom.
I know for the serial port there were libraries that existed for interacting with it (rs232 library). Googling for java and USB returned several answers (the first was called jUSB). That would be the first type of thing I would be looking for.
sample for java usb connection to freescale microcontroller:
http://javausbapi.blogspot.com/
I want to build a community website with videoconferencing functionality integrated. I would prefer to provide this from within the browser, so I'm looking for a Java- or Flash-based solution.
Also, it would be nice to spare bandwidth by having the clients stream their audio and video data without using a central server (like the way Skype works, for example).
Is there a reasonably mature open-source project that meets these criteria?
When doing Voice over IP, UDP is preferred, and web browser plugins usually lacks support for UDP and P2P-connections.
But there is a solution. You have to use Adobe Flash Player 10 and the Adobe Stratus for this. This is actually what Chatroulette is using for P2P streamed video chat.
Take it from someone who found out the hard way: Java's support for video playback is pisspoor. They keep promising a new, fantastic video playback system, but meanwhile we are STILL stuck with Java Media Framework, the platform-limited, MPEG-4 incompatible mess.
Your alternatives are Quicktime for Java and systems wrapping MPlayer/VLC/etc. None of those are really options for a browser.
Much as I hate Adobe and the evils of Flash site design, Flash is far and away your best option.
See the Xuggler library -- it includes code for playing back video (well, sample code) in the com.xuggle.mediatool.IMediaVieer object.