Develop Reference

Google Cloud Speech To Text Giving 0 result

I'm using Google Cloud Speech to text api in Java.
I'm getting 0 results when I call speechClient.recognize
pom.xml:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-speech</artifactId>
<version>0.80.0-beta</version>
</dependency>
Java code:
import java.io.FileInputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.speech.v1.RecognitionAudio;
import com.google.cloud.speech.v1.RecognitionConfig;
import com.google.cloud.speech.v1.RecognitionConfig.AudioEncoding;
import com.google.cloud.speech.v1.RecognizeResponse;
import com.google.cloud.speech.v1.SpeechClient;
import com.google.cloud.speech.v1.SpeechRecognitionAlternative;
import com.google.cloud.speech.v1.SpeechRecognitionResult;
import com.google.cloud.speech.v1.SpeechSettings;
import com.google.protobuf.ByteString;
public class SpeechToText {
public static void main(String[] args) {
// Instantiates a client
try {
String jsonFilePath = System.getProperty("user.dir") + "/serviceaccount.json";
FileInputStream credentialsStream = new FileInputStream(jsonFilePath);
GoogleCredentials credentials = GoogleCredentials.fromStream(credentialsStream);
FixedCredentialsProvider credentialsProvider = FixedCredentialsProvider.create(credentials);
SpeechSettings speechSettings =
SpeechSettings.newBuilder()
.setCredentialsProvider(credentialsProvider)
.build();
SpeechClient speechClient = SpeechClient.create(speechSettings);
//SpeechClient speechClient = SpeechClient.create();
// The path to the audio file to transcribe
String fileName = System.getProperty("user.dir") + "/call-recording-790.opus";
// Reads the audio file into memory
Path path = Paths.get(fileName);
byte[] data = Files.readAllBytes(path);
ByteString audioBytes = ByteString.copyFrom(data);
System.out.println(path.toAbsolutePath());
// Builds the sync recognize request
RecognitionConfig config = RecognitionConfig.newBuilder().setEncoding(AudioEncoding.LINEAR16)
.setSampleRateHertz(8000).setLanguageCode("en-US").build();
RecognitionAudio audio = RecognitionAudio.newBuilder().setContent(audioBytes).build();
System.out.println("recognize builder");
// Performs speech recognition on the audio file
RecognizeResponse response = speechClient.recognize(config, audio);
List<SpeechRecognitionResult> results = response.getResultsList();
System.out.println(results.size()); // ***** HERE 0
for (SpeechRecognitionResult result : results) {
// There can be several alternative transcripts for a given chunk of speech.
// Just use the
// first (most likely) one here.
SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
System.out.printf("Transcription: %s%n", alternative.getTranscript());
}
} catch (Exception e) {
System.out.println(e);
}
}
}
In the code above, I'm getting results.size as 0. When I upload the same opus file on demo at https://cloud.google.com/speech-to-text/, it gives output text correctly.
So why is the recognize call giving zero results?

There could be 3 reasons for Speech-to-Text to return an empty response:
Audio is not clear.
Audio is not intelligible.
Audio is not using the proper encoding.
From what I can see, reason 3 is the most possible cause of your issue. To resolve this, check this page to know how to verify the encoding of your audio file which must match the parameters you sent in InitialRecognizeRequest.

Google Text to Speech played with my speaker

I would like a text as input to reproduce the content of that text as a synthesized voice with the speakers of my computer. The application I'm developing is in Java and to get the desired result I'm using the Google-test-to-speech libraries. I started with the following code:
https://cloud.google.com/text-to-speech/docs/reference/libraries
but as you can see the voice message is saved on file and not reproduced with the speakers. So my question is how can I reproduce the voice message with the speakers of the computer running the application?
import com.google.cloud.texttospeech.v1.AudioEncoding;
import com.google.cloud.texttospeech.v1.SsmlVoiceGender;
import com.google.cloud.texttospeech.v1.SynthesisInput;
import com.google.cloud.texttospeech.v1.SynthesizeSpeechResponse;
import com.google.cloud.texttospeech.v1.TextToSpeechClient;
import com.google.cloud.texttospeech.v1.VoiceSelectionParams;
import com.google.protobuf.ByteString;
import java.io.FileOutputStream;
import java.io.OutputStream;
public class QuickstartSample {
public static void main(String... args) throws Exception {
// Instantiates a client
try (TextToSpeechClient textToSpeechClient = TextToSpeechClient.create()) {
// Set the text input to be synthesized
SynthesisInput input = SynthesisInput.newBuilder()
.setText("Hello, World!")
.build();
// Build the voice request, select the language code ("en-US") and the ssml voice gender
// ("neutral")
VoiceSelectionParams voice = VoiceSelectionParams.newBuilder()
.setLanguageCode("en-US")
.setSsmlGender(SsmlVoiceGender.NEUTRAL)
.build();
// Select the type of audio file you want returned
AudioConfig audioConfig = AudioConfig.newBuilder()
.setAudioEncoding(AudioEncoding.MP3)
.build();
// Perform the text-to-speech request on the text input with the selected voice parameters and
// audio file type
SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice,
audioConfig);
// Get the audio contents from the response
ByteString audioContents = response.getAudioContent();
// Write the response to the output file.
try (OutputStream out = new FileOutputStream("output.mp3")) {
out.write(audioContents.toByteArray());
System.out.println("Audio content written to file \"output.mp3\"");
}
}
}
}

You can convert the audio content to an InputStream and then use javazoom Player to play it
import javazoom.jl.decoder.JavaLayerException;
import javazoom.jl.player.Player;
SynthesizeSpeechResponse response = textToSpeechClient.synthesizeSpeech(input, voice,audioConfig);
InputStream targetStream = new ByteArrayInputStream(response.getAudioContent().toByteArray());
Player playMP3;
playMP3 = new Player(targetStream);
playMP3.play();

How do you read and print a chunked HTTP response using java.net.http as chunks arrive?

Java 11 introduces a new package, java.net.http, for making HTTP requests. For general usage, it's pretty straight forward.
My question is: how do I use java.net.http to handle chunked responses as each chunk is received by the client?
java.http.net contains a reactive BodySubscriber which appears to be what I want, but I can't find an example of how it's used.
http_get_demo.py
Below is a python implementation that prints chunks as they arrive, I'd like to the same thing with java.net.http:
import argparse
import requests
def main(url: str):
with requests.get(url, stream=True) as r:
for c in r.iter_content(chunk_size=1):
print(c.decode("UTF-8"), end="")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Read from a URL and print as text as chunks arrive")
parser.add_argument('url', type=str, help="A URL to read from")
args = parser.parse_args()
main(args.url)
HttpGetDemo.java
Just for completeness, here's a simple example of making a blocking request using java.net.http:
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpResponse;
import java.net.http.HttpRequest;
public class HttpGetDemo {
public static void main(String[] args) throws Exception {
var request = HttpRequest.newBuilder()
.uri(URI.create(args[0]))
.build();
var bodyHandler = HttpResponse.BodyHandlers
.ofString();
var client = HttpClient.newHttpClient();
var response = client.send(request, bodyHandler);
System.out.println(response.body());
}
}
HttpAsyncGetDemo.java
And here's the example making an non-blocking/async request:
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpResponse;
import java.net.http.HttpRequest;
/**
* ReadChunked
*/
public class HttpAsyncGetDemo {
public static void main(String[] args) throws Exception {
var request = HttpRequest.newBuilder()
.uri(URI.create(args[0]))
.build();
var bodyHandler = HttpResponse.BodyHandlers
.ofString();
var client = HttpClient.newHttpClient();
client.sendAsync(request, bodyHandler)
.thenApply(HttpResponse::body)
.thenAccept(System.out::println)
.join();
}
}

The python code does not ensure that the response body data is made available one HTTP chunk at a time. It just provides small amounts of data to the application, thus reducing the amount of memory consumed at the application level ( it could be buffered lower in the stack ). The Java 11 HTTP Client supports streaming through one of the streaming body handlers, HttpResponse.BodyHandlers: ofInputStream, ofByteArrayConsumer, asLines, etc.
Or write your own handler / subscriber as demonstrated:
https://www.youtube.com/watch?v=qiaC0QMLz5Y

You can print ByteBuffers as they come, but there's no guarantee that a ByteBuffer corresponds to a chunk. Chunks are handled by the stack. One ByteBuffer slice will be pushed for every chunk - but if there isn’t enough space remaining in the buffer, then a partial chunk will be pushed. All the consumer sees is a stream of ByteBuffers that contain the data.
So what you can do is print those ByteBuffers as they come, but you have no guarantee that they correspond exactly one chunk each as was sent by the server.
Note: If the body of your request is text based, then you can use
BodyHandlers.fromLineSubscriber(Subscriber<? super String> subscriber) with a custom Subscriber<String> that will print each line as it comes.
The BodyHandlers.fromLineSubscriber does the hard word of decoding bytes into chars using the charset indicated in the response headers, buffering bytes if needed until they can be decoded (a ByteBuffer might end in the middle of an encoding sequence if the text contains chars encoded over multiple bytes), and splitting them at the line boundary. The Subscriber::onNext method will be invoked once for each line in the text. See https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/HttpResponse.BodyHandlers.html#fromLineSubscriber(java.util.concurrent.Flow.Subscriber) for more info.

Thanks to #pavel and #chegar999 for their partial answers. They led me to my solution.
Overview
The solution I came up with is below. Basically, the solution is to use a custom java.net.http.HttpResponse.BodySubscriber. A BodySubscriber contains reactive methods (onSubscribe, onNext, onError, and onComplete) and a getBody method that basically returns a java CompletableFuture that will eventually produce the body of the HTTP request. Once you have your BodySubscriber in hand you can use it like:
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(uri))
.build();
return client.sendAsync(request, responseInfo -> new StringSubscriber())
.whenComplete((r, t) -> System.out.println("--- Status code " + r.statusCode()))
.thenApply(HttpResponse::body);
Note the line:
client.sendAsync(request, responseInfo -> new StringSubscriber())
That's where we register our custom BodySubscriber; in this case, my custom class is named StringSubscriber.
CustomSubscriber.java
This is a complete working example. Using Java 11, you can run it without compiling it. Just past it into a file named CustomSubscriber.java, then run the command java CustomSubscriber <some url>. It prints the contents of each chunk as it arrives. It also collects them and returns them as the body when the response has completed.
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpResponse.BodyHandlers;
import java.net.http.HttpResponse.BodySubscriber;
import java.net.URI;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.Flow;
import java.util.stream.Collectors;
import java.util.List;
public class CustomSubscriber {
public static void main(String[] args) {
CustomSubscriber cs = new CustomSubscriber();
String body = cs.get(args[0]).join();
System.out.println("--- Response body:\n: ..." + body + "...");
}
public CompletableFuture<String> get(String uri) {
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(uri))
.build();
return client.sendAsync(request, responseInfo -> new StringSubscriber())
.whenComplete((r, t) -> System.out.println("--- Status code " + r.statusCode()))
.thenApply(HttpResponse::body);
}
static class StringSubscriber implements BodySubscriber<String> {
final CompletableFuture<String> bodyCF = new CompletableFuture<>();
Flow.Subscription subscription;
List<ByteBuffer> responseData = new CopyOnWriteArrayList<>();
#Override
public CompletionStage<String> getBody() {
return bodyCF;
}
#Override
public void onSubscribe(Flow.Subscription subscription) {
this.subscription = subscription;
subscription.request(1); // Request first item
}
#Override
public void onNext(List<ByteBuffer> buffers) {
System.out.println("-- onNext " + buffers);
try {
System.out.println("\tBuffer Content:\n" + asString(buffers));
}
catch (Exception e) {
System.out.println("\tUnable to print buffer content");
}
buffers.forEach(ByteBuffer::rewind); // Rewind after reading
responseData.addAll(buffers);
subscription.request(1); // Request next item
}
#Override
public void onError(Throwable throwable) {
bodyCF.completeExceptionally(throwable);
}
#Override
public void onComplete() {
bodyCF.complete(asString(responseData));
}
private String asString(List<ByteBuffer> buffers) {
return new String(toBytes(buffers), StandardCharsets.UTF_8);
}
private byte[] toBytes(List<ByteBuffer> buffers) {
int size = buffers.stream()
.mapToInt(ByteBuffer::remaining)
.sum();
byte[] bs = new byte[size];
int offset = 0;
for (ByteBuffer buffer : buffers) {
int remaining = buffer.remaining();
buffer.get(bs, offset, remaining);
offset += remaining;
}
return bs;
}
}
}
Trying it out
To test this solution, you'll need a server that sends a response that uses Transfer-encoding: chunked and sends it slow enough to watch the chunks arrive. I've created one at https://github.com/hohonuuli/demo-chunk-server but you can spin it up using Docker like so:
docker run -p 8080:8080 hohonuuli/demo-chunk-server
Then run the CustomSubscriber.java code using java CustomSubscriber.java http://localhost:8080/chunk/10

There is now a new Java library to address this kind of requirements
RxSON: https://github.com/rxson/rxson
It utilizes the JsonPath wit RxJava to read JSON streamed chunks from the response as soon as they arrive, and parse them to java objects.
Example:
String serviceURL = "https://think.cs.vt.edu/corgis/datasets/json/airlines/airlines.json";
HttpRequest req = HttpRequest.newBuilder(URI.create(serviceURL)).GET().build();
RxSON rxson = new RxSON.Builder().build();
String jsonPath = "$[*].Airport.Name";
Flowable<String> airportStream = rxson.create(String.class, req, jsonPath);
airportStream
.doOnNext(it -> System.out.println("Received new item: " + it))
//Just for test
.toList()
.blockingGet();

How can I send a PNG of a QR-code in a HTTP response body (with Spark)?

I want to generate a QR-code image, convert it to PNG and return it as a HTTP response to my client.
To generate the QR-code I use ZXing. I already tested the conversion part by writing using a FileOutputStream with MatrixToImageWriter.writeToStream(...). That works like a charm.
The web framework I am currently using is Spark (Version 1.1.1). The return of the handle(...)-method is set as the response body. What am I doing wrong here?
With the current solution I get The image "http://localhost:4567/qrcode" cannot be displayed because it contains errors when performing the GET-request with Firefox.
import java.io.ByteArrayOutputStream;
import java.io.OutputStream;
import static spark.Spark.get;
import spark.Request;
import spark.Response;
import spark.Route;
import com.google.gson.Gson;
import com.google.common.io.BaseEncoding;
import com.google.zxing.BarcodeFormat;
import com.google.zxing.client.j2se.MatrixToImageWriter;
import com.google.zxing.common.BitMatrix;
import com.google.zxing.qrcode.QRCodeWriter;
public class App {
public static void main(String[] args) {
get(new Route("/qrcode") {
#Override
public Object handle(Request request, Response response) {
// Test data
QrData data = new QrData("test");
// Data is wrapped in JSON
String json = new Gson().toJson(data);
// Transform JSON to QR-code PNG byte string
String qrString = "";
try {
qrString = generatePngQrCode(json);
} catch (Exception e) {
e.printStackTrace();
}
// Set response parameters
response.type("image/png");
response.status(200);
// Return response body
return qrString;
}
});
}
public String generatePngQrCode(String content) throws Exception {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
// ZXing QR-code encoding
BitMatrix bitMatrix = new QRCodeWriter().encode(content, BarcodeFormat.QR_CODE, 400, 400);
// Convert to PNG image and write to stream
MatrixToImageWriter.writeToStream(bitMatrix, "png", outputStream);
// Encode to Base 64
return BaseEncoding.base64().encode(outputStream.toByteArray());
}
}

Just went through this. You can write any file/binary data/output stream using the following code:
byte[] bytes = Files.readAllBytes(Paths.get(filePath));
HttpServletResponse raw = res.raw();
raw.getOutputStream().write(bytes);
raw.getOutputStream().flush();
raw.getOutputStream().close();
return res.raw();

Use response.getRaw to obtain an OutputStream that should be used to write the PNG to (using MatrixToImageWriter).

returning base64 string from Java

Very Basically I have an image I have converted into a Base64 string. The following code is how it is turned into a base64string and how im trying to return it. It is a modification of a phonegap plugin so please excuse the naming of certain things.
package org.apache.cordova;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.cordova.api.Plugin;
import org.apache.cordova.api.PluginResult;
import org.json.JSONArray;
import android.annotation.TargetApi;
import android.graphics.Bitmap;
import android.os.Environment;
import android.util.Base64;
import android.view.View;
public class Screenshot extends Plugin {
#Override
public PluginResult execute(String action, JSONArray args, String callbackId) {
// starting on ICS, some WebView methods
// can only be called on UI threads
final Plugin that = this;
final String id = callbackId;
super.cordova.getActivity().runOnUiThread(new Runnable() {
//#Override
#TargetApi(8)
public void run() {
View view = webView.getRootView();
view.setDrawingCacheEnabled(true);
Bitmap bitmap = Bitmap.createBitmap(view.getDrawingCache());
view.setDrawingCacheEnabled(false);
File folder = new File(Environment.getExternalStorageDirectory(), "Pictures");
if (!folder.exists()) {
folder.mkdirs();
}
File f = new File(folder, "screenshot_" + System.currentTimeMillis() + ".png");
System.out.println(folder);
System.out.println("screenshot_" + System.currentTimeMillis() + ".png");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos);
byte[] b = baos.toByteArray();
String base64String = Base64.encodeToString(b, Base64.DEFAULT);
String mytextstring = "data:image/png;base64,"+base64String;
System.out.println(mytextstring);
that.success(new PluginResult(PluginResult.Status.OK), mytextstring);
}
});
PluginResult imageData = new PluginResult(PluginResult.Status.NO_RESULT);
imageData.setKeepCallback(true);
System.out.println("imageData:==================================================================================>>>>>"+imageData);
return imageData;
}
}
This line:
System.out.println(mytextstring);
outputs the base64 string, however this line:
System.out.println("imageData:==================================================================================>>>>>"+imageData);
doesn't. I'm trying to get imageData to contain the base64string to I can return that and use it in some Javascript. i have played about with the code for a while now with no success, I am new to Java so really don't know how close I am. Can anyone guide me please?
EDIT Simply all I want to do is return the Base64String, nothing else.

Your code that calls this.success is incorrect. It should be:
that.success(new PluginResult(PluginResult.Status.OK, mytextstring), id);
The way you have it:
that.success(new PluginResult(PluginResult.Status.OK), mytextstring);
Tells the callback server to call the success callback at "mytextstring". Obviously there is no callback stored at "mytextstring" as that is your result.
In addition the default .js file for the screen shot plugin does not provide a way for you to call it with success/failure callbacks. So you will need to modify the .js code in order to get a success callback invoked with your base64 data. Add something like this to Screenshot.js:
Screenshot.prototype.getBase64Screenshot = function(successCallback, errorCallback) {
exec(successCallback, errorCallback, "Screenshot", "saveScreenshot", []);
};
Then you can call it by doing this:
window.plugins.screenshot.getBase64Screenshot(function(data) {
console.log("Data = " + data);
}, function() {
console.log("We got an error");
});
Obviously you'd provide your own success and failure callbacks.

I think its a mistake in your code. I'll try to provide a couple of ways from the point of view of pure java
When you create your 'PluginResult' object you don't connect it in any manner to the string you've created (base64string variable).
If you want to return the base64-encoded string you should do something like:
PluginResult imageData = new PluginResult(..., ..., base64string, ..., ...)
or if you don't have a constructor try to use setter like this (for example):
PlugingResult imageData = new PlugingResult(...)
imageData.setEncodedString(imageData)
Without an exposed API its hard to understand how to use object of type PluginResult properly (I've never used it by myself)
If you see that nothing suits your needs but you absolutely have to use the PluginResult object, than you may consider to create a class that contains both PluginResult and your encoded String and use it as a return type of your method. Like this:
class MyPluginResult {
private PluginResult pluginResult;
private String encodedString;
/*
....
getters/setters/constructors go here
*/
}
Than make your method returning MyPluginResult, change the code that creates a PluginResult and returns it and you'll be fine.
Hope this helps

I dont know much about Cordova and PhoneGap, but looking through the API you have the constructor public PluginResult(Status status, String message). I guess you can pass the base64String to the second parameter of that constructor. The message is converted to a JSON string format, but you can handle that format in java / javascript easily.
Hope this helps.
Regards