Develop Reference

Java Microsoft Azure speech-to-text from bytes?

I am thinking about building the code more efficiently. I am using Discord JDA and the Microsoft Azure speech service. Is it possible to recognize speech directly from bytes, not from a file? I mean, skipping writing bytes to a temporary file and then recognizing the file. Or maybe in some other, better way I can do it? The current method seems inappropriate to me.
My AudioReceiveHandler:
#Override
public void handleUserAudio(#NotNull UserAudio userAudio) {
User user = userAudio.getUser();
if (!BYTES.containsKey(user))
BYTES.put(user, new ArrayList<>());
ArrayList<byte[]> userBytes = BYTES.get(user);
userBytes.add(userAudio.getAudioData(1));
}
Converting speech to text:
private void read(ArrayList<byte[]> userBytes) {
int length = 0;
for (byte[] bytes : userBytes) {
length += bytes.length;
}
byte[] decodedData = new byte[length];
int i = 0;
for (byte[] bytes : userBytes) {
for (byte sampleByte : bytes) {
decodedData[i++] = sampleByte;
}
}
String filePath = "[...]/temp.wav";
try {
AudioSystem.write(new AudioInputStream(new ByteArrayInputStream(decodedData),
new AudioFormat(48000f, 16, 2, true, true), decodedData.length),
AudioFileFormat.Type.WAVE,
new File(filePath));
} catch (IOException exception) {
exception.printStackTrace();
}
SpeechConfig speechConfig = SpeechConfig.fromSubscription("-", "-");
speechConfig.setSpeechRecognitionLanguage("pl-PL");
AudioConfig audioConfig = AudioConfig.fromWavFileInput(filePath);
SpeechRecognizer recognizer = new SpeechRecognizer(speechConfig, audioConfig);
Future<SpeechRecognitionResult> task = recognizer.recognizeOnceAsync();
try {
SpeechRecognitionResult result = task.get();
Logger.info("RECOGNIZED: " + result.getText());
} catch (Exception exception) {
exception.printStackTrace();
}
}

cannot convert string to int after receiving data in java

i have arduino that send 2 int and the int was separate by " : "
for example my arduino send data = 12315: 15123
after i send that data using bluetooth, i receive the data in smartphone and i need that 2 value that i receive become int again
this is was my part code on my android
void beginListenForData()
{
final Handler handler = new Handler();
final byte delimiter = 10; //This is the ASCII code for a newline character
stopWorker = false;
readBufferPosition = 0;
readBuffer = new byte[1024];
workerThread = new Thread(new Runnable()
{
public void run()
{
while(!Thread.currentThread().isInterrupted() && !stopWorker)
{
try
{
int bytesAvailable = mmInputStream.available();
if(bytesAvailable > 0)
{
byte[] packetBytes = new byte[bytesAvailable];
mmInputStream.read(packetBytes);
for(int i=0;i<bytesAvailable;i++)
{
byte b = packetBytes[i];
if(b == delimiter)
{
byte[] encodedBytes = new byte[readBufferPosition];
System.arraycopy(readBuffer, 0, encodedBytes, 0, encodedBytes.length);
final String data = new String(encodedBytes, "US-ASCII");
readBufferPosition = 0;
handler.post(new Runnable()
{
public void run()
{
String[] tmp;
String lol,dam;
tmp = data.split(":");
try
{
lol = String.valueOf(crypt[0])+":"+String.valueOf(crypt[1]);
derajat.setText(lol);
}
catch (NumberFormatException nfe)
{
// bad data - set to sentinel
crypt[0] = Integer.MIN_VALUE;
crypt[1] = Integer.MIN_VALUE;
derajat.setText("none");
}
dam = tmp[0]+":"+tmp[1];
myLabel.setText(dam);
//derajat.setText(data);
}
});
}
else
{
readBuffer[readBufferPosition++] = b;
}
}
}
}
catch (IOException ex)
{
stopWorker = true;
}
}
}
});
workerThread.start();
}
as you can see that after '\n' data will be process
first i split using data.split() and i get 2 value that stored in tmp
after that i parse that value to crypt[0] and crypt[1]
i set text to derajat to see my value
and i set text the result in myLabel too
i can see my value in myLabel but i cant convert it to int because i see nothing in label derajat
can somebody help me why i cant convert int after receiving that data ??

If the received data from your Arduino is in fact as you have specified:
"12315: 15123"
Then what is happening is that you are getting a NumberFormatException which you do catch but really have no way of knowing other than derajat will contain nothing (or "none")....which is exactly what you are experiencing.
The reason you would get a NumberFormatException is because there is a white-space directly after the colon (:) delimiter so when you try to carry out:
crypt[1] = Integer.parseInt(tmp[1]);
(tmp[1] actually holds " 15123") a NumberFormatException is thrown. The Integer.parseInt() method will not handle white-spaces within the numerical string. To remedy this trim your split strings, for example:
crypt[0] = Integer.parseInt(tmp[0].trim());
crypt[1] = Integer.parseInt(tmp[1].trim());
or split with:
tmp = data.split(": ");

Receive string via Bluetooth android (without line feed or Carriage Return)

There are a lot of questions on stack overflow on how to receive data (string) from a Bluetooth device but I have never seen a question on how to receive data (string) without checking for LF or CR.
This code works perfectly when the device send a LF at the end of the String:
final Handler handler = new Handler();
final byte delimiter = 10; //ASCII code for Line Feed
stopWorker = false;
readBufferPosition = 0;
readBuffer = new byte[1024];
try {
inStream = btSocket.getInputStream();
} catch (IOException e) {
Toast.makeText(getApplicationContext(), e.toString(), Toast.LENGTH_SHORT).show();
}
Thread workerThread = new Thread(new Runnable() {
#Override
public void run() {
while (!Thread.currentThread().isInterrupted() && !stopWorker) {
try {
int bytesAvailable = inStream.available();
if (bytesAvailable > 0) {
byte[] packetBytes = new byte[bytesAvailable];
inStream.read(packetBytes);
for (int i = 0; i < bytesAvailable; i++) {
byte b = packetBytes[i];
if (b == delimiter) {
byte[] encodedBytes = new byte[readBufferPosition];
System.arraycopy(readBuffer, 0, encodedBytes, 0, encodedBytes.length);
final String data = new String(encodedBytes, "US-ASCII");
readBufferPosition = 0;
handler.post(new Runnable() {
public void run() {
//test
textview1.setText(data);
}
});
} else {
readBuffer[readBufferPosition++] = b;
}
}
}
} catch (IOException ex) {
stopWorker = true;
}
}
}
});
workerThread.start();
But the device that sends the string does not send a Line feed neither a CR at the end. I have tried many things (also the Bluetooth guide from Google (http://developer.android.com/guide/topics/connectivity/bluetooth.html)). I want to receive all the data that is send to the app (and for testing display the received String in a textview).
Thanks in advance,
Michielvk

Assuming all strings start with a "1" character, you should be able to read the second to last string by replacing final byte delimiter = 10; by final byte delimiter = 49; (since 49 is decimal representation of "1" in ASCII table).
There's no way to read the last string (and assert you're reading the full string, not only a part) if there's no suffix delimiter.
EDIT :
You gave the protocol's documentation in this comment feed. The doc says :
The result string [...] consists of 38 ASCII characters.
According to the fact that each response consists of 38 ASCII characters, you should be able to read all messages by replacing :
if (b == delimiter) {
with :
if (readBufferPosition >= 37) {

Capturing Buffering playing live Audio Streaming

I am getting live audio streaming over the network in the form of RTP packets and I have to write a code to Capture, Buffer and play the audio stream.
Problem
Now to solve this problem I have written two threads one for capture the audio and another for playing it. Now when I start both the threads my capture threads running slower than playing thread :(
Buffer Requirement
RTP Audio Packets.
8kHz, 16-bit Linear Samples (Linear PCM).
4 frames of 20ms audio will be sent in each RTP Packet.
Do not play until AudioStart=24 (# of 20ms frames) have arrived.
While playing ... if the # of 20ms frames in buffer reaches 0 ...
stop playing until AudioStart frames are buffered then restart.
While playing ... if the # of 20ms frames in buffer exceeds
AudioBufferHigh=50 then delete 24 frames (in easiest manner -- delete
from buffer or just drop next 6 RTP messages).
What I have done so far..
Code
BufferManager.java
public abstract class BufferManager {
protected static final Integer ONE = new Integer(1);
protected static final Integer TWO = new Integer(2);
protected static final Integer THREE = new Integer(3);
protected static final Integer BUFFER_SIZE = 5334;//5.334KB
protected static volatile Map<Integer, ByteArrayOutputStream> bufferPool = new ConcurrentHashMap<>(3, 0.9f, 2);
protected static volatile Integer captureBufferKey = ONE;
protected static volatile Integer playingBufferKey = ONE;
protected static Boolean running;
protected static volatile Integer noOfFrames = 0;
public BufferManager() {
//captureBufferKey = ONE;
//playingBufferKey = ONE;
//noOfFrames = new Integer(0);
}
protected void switchCaptureBufferKey() {
if(ONE.intValue() == captureBufferKey.intValue())
captureBufferKey = TWO;
else if(TWO.intValue() == captureBufferKey.intValue())
captureBufferKey = THREE;
else
captureBufferKey = ONE;
//printBufferState("SWITCHCAPTURE");
}//End of switchWritingBufferKey() Method.
protected void switchPlayingBufferKey() {
if(ONE.intValue() == playingBufferKey.intValue())
playingBufferKey = TWO;
else if(TWO.intValue() == playingBufferKey.intValue())
playingBufferKey = THREE;
else
playingBufferKey = ONE;
}//End of switchWritingBufferKey() Method.
protected static AudioFormat getFormat() {
float sampleRate = 8000;
int sampleSizeInBits = 16;
int channels = 1;
boolean signed = true;
boolean bigEndian = true;
return new AudioFormat(sampleRate, sampleSizeInBits, channels, signed, bigEndian);
}
protected int getByfferSize() {
return bufferPool.get(ONE).size()
+ bufferPool.get(TWO).size()
+ bufferPool.get(THREE).size();
}
protected static void printBufferState(String flag) {
int a = bufferPool.get(ONE).size();
int b = bufferPool.get(TWO).size();
int c = bufferPool.get(THREE).size();
System.out.println(flag + " == TOTAL : [" + (a + b +c) + "bytes] ");
// int a,b,c;
// System.out.println(flag + "1 : [" + (a = bufferPool.get(ONE).size()) + "bytes], 2 : [" + (b = bufferPool.get(TWO).size())
// + "bytes] 3 : [" + (c = bufferPool.get(THREE).size()) + "bytes], TOTAL : [" + (a + b +c) + "bytes] ");
}
}//End of BufferManager Class.
AudioCapture.java
public class AudioCapture extends BufferManager implements Runnable {
private static final Integer RTP_HEADER_SIZE = 12;
private InetAddress ipAddress;
private DatagramSocket serverSocket;
long lStartTime = 0;
public AudioCapture(Integer port) throws UnknownHostException, SocketException {
super();
running = Boolean.TRUE;
bufferPool.put(ONE, new ByteArrayOutputStream(BUFFER_SIZE));
bufferPool.put(TWO, new ByteArrayOutputStream(BUFFER_SIZE));
bufferPool.put(THREE, new ByteArrayOutputStream(BUFFER_SIZE));
this.ipAddress = InetAddress.getByName("0.0.0.0");
serverSocket = new DatagramSocket(port, ipAddress);
}
#Override
public void run() {
System.out.println();
byte[] receiveData = new byte[1300];
DatagramPacket receivePacket = null;
lStartTime = System.currentTimeMillis();
receivePacket = new DatagramPacket(receiveData, receiveData.length);
byte[] packet = new byte[receivePacket.getLength() - RTP_HEADER_SIZE];
ByteArrayOutputStream buff = bufferPool.get(captureBufferKey);
while (running) {
if(noOfFrames <= 50) {
try {
serverSocket.receive(receivePacket);
packet = Arrays.copyOfRange(receivePacket.getData(), RTP_HEADER_SIZE, receivePacket.getLength());
if((buff.size() + packet.length) > BUFFER_SIZE) {
switchCaptureBufferKey();
buff = bufferPool.get(captureBufferKey);
}
buff.write(packet);
noOfFrames += 4;
} catch (SocketException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} // End of try-catch block.
} else {
//System.out.println("Packet Ignored, Buffer reached to its maximum limit ");
}//End of if-else block.
} // End of while loop.
}//End of run() Method.
}
AudioPlayer.java
public class AudioPlayer extends BufferManager implements Runnable {
long lStartTime = 0;
public AudioPlayer() {
super();
}
#Override
public void run() {
AudioFormat format = getFormat();
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
SourceDataLine line = null;
try {
line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
} catch (LineUnavailableException e1) {
e1.printStackTrace();
}
while (running) {
if (noOfFrames >= 24) {
ByteArrayOutputStream out = null;
try {
out = bufferPool.get(playingBufferKey);
InputStream input = new ByteArrayInputStream(out.toByteArray());
byte buffer[] = new byte[640];
int count;
while ((count = input.read(buffer, 0, buffer.length)) != -1) {
if (count > 0) {
InputStream in = new ByteArrayInputStream(buffer);
AudioInputStream ais = new AudioInputStream(in, format, buffer.length / format.getFrameSize());
byte buff[] = new byte[640];
int c = 0;
if((c = ais.read(buff)) != -1)
line.write(buff, 0, buff.length);
}
}
} catch (IOException e) {
e.printStackTrace();
}
/*byte buffer[] = new byte[1280];
try {
int count;
while ((count = ais.read(buffer, 0, buffer.length)) != -1) {
if (count > 0) {
line.write(buffer, 0, count);
}
}
} catch (IOException e) {
e.printStackTrace();
}*/
out.reset();
noOfFrames -= 4;
try {
if (getByfferSize() >= 10240) {
Thread.sleep(15);
} else if (getByfferSize() >= 5120) {
Thread.sleep(25);
} else if (getByfferSize() >= 0) {
Thread.sleep(30);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
// System.out.println("Number of frames :- " + noOfFrames);
}
}
}// End of run() method.
}// End of AudioPlayer Class class.
any help or pointer to the helpful link will be appreciable Thanks...

This answer explains a few challenges with streaming.
In a nutshell, your client needs to deal with two issues:
1) The clock (crystals) on the client and server are not perfectly in sync. The server may be a fraction of a Hz faster/slower than the client. The client continuously match the infer the clock rate of the server by examining the rate that rtp packets are delivered. The client then adjusts the playback rate via sample rate conversion. So instead of playing back at 48k, it may play back at 48000.0001 Hz.
2) Packets loss, out of order arrivals, etc. must be dealt with. If you lose packets, you need to still keep a place holder for those packets in your buffer stream otherwise your audio will skip and sound crackly and become unaligned. The simplest method would be to replace those missing packets with silence but the volume of adjacent packets should be adjusted to avoid sharp envelope changes snapping to 0.
Your design seems a bit unorthodox. I have had success using a ring buffer instead. You will have to deal with edge cases as well.
I always state that streaming media is not a trivial task.

Buffered Reader for Android Streaming from Bluetooth

Hey guys so I'm trying to read a stream from a bluetooth device continuously streaming integers like this:
-11
121
123
1234
-11
I have everything working with some code I found online, but to do some of the processing the numbers need to be ints as opposed to Strings, parseInt is taking up too much CPU, and I tried using a buffered stream with no avail.
Here is the current method:
void beginListenForData()
{
final Handler handler = new Handler();
final byte delimiter = 10; //This is the ASCII code for a newline character
stopWorker = false;
readBufferPosition = 0;
readBuffer = new byte[1024];
workerThread = new Thread(new Runnable()
{
public void run()
{
while(!Thread.currentThread().isInterrupted() && !stopWorker)
{
try
{
int bytesAvailable = mmInputStream.available();
if(bytesAvailable > 0)
{
byte[] packetBytes = new byte[bytesAvailable];
mmInputStream.read(packetBytes);
for(int i=0;i<bytesAvailable;i++)
{
byte b = packetBytes[i];
if(b == delimiter)
{
byte[] encodedBytes = new byte[readBufferPosition];
System.arraycopy(readBuffer, 0, encodedBytes, 0, encodedBytes.length);
final String data = new String(encodedBytes, "US-ASCII");
readBufferPosition = 0;
handler.post(new Runnable()
{
public void run()
{
myLabel.setText(data);
}
});
}
else
{
readBuffer[readBufferPosition++] = b;
}
}
}
}
catch (IOException ex)
{
stopWorker = true;
}
}
}
});
workerThread.start();
}
If it makes a difference, the data is coming from an Arudino, which I can modify the way it streams.
Thanks!

Use a DataInputStream wrapped around a BufferedInputStream, and the readInt() method. This assumes network byte order in the integers of course.
Forget all this arraycopy() stuff.