I've been trying to get the Google Pub/Sub Java libraries to work using the Quickstart guides. None of them work as written, at least not for me. I'm working in IntelliJ, Maven framework, OSX, Java 8.
Take this example. I followed all the steps: Created a service account as Project Owner, installed the Gcloud SDK, set my GOOGLE_APPLICATIONS_CREDENTIALS envvar, and found that everything was hunky dory from the command line: I could create topics, publish messages, whatever I wanted.
Then when I tried to run the sample code:
import com.google.api.gax.rpc.ApiException;
import com.google.cloud.ServiceOptions;
import com.google.cloud.pubsub.v1.TopicAdminClient;
import com.google.pubsub.v1.ProjectTopicName;
public class CreateTopicExample {
/**
* Create a topic.
*
* #param args topicId
* #throws Exception exception thrown if operation is unsuccessful
*/
public static void main(String... args) throws Exception {
// Your Google Cloud Platform project ID
String projectId = ServiceOptions.getDefaultProjectId();
// Your topic ID, eg. "my-topic"
String topicId = args[0];
// Create a new topic
ProjectTopicName topic = ProjectTopicName.of(projectId, topicId);
try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {
topicAdminClient.createTopic(topic);
} catch (ApiException e) {
// example : code = ALREADY_EXISTS(409) implies topic already exists
System.out.print(e.getStatusCode().getCode());
System.out.print(e.isRetryable());
}
System.out.printf("Topic %s:%s created.\n", topic.getProject(), topic.getTopic());
}
}
It throws the ApiException PERMISSION_DENIED. The full stacktrace:
com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: User not authorized to perform this action.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:55)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:95)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:61)
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1123)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:492)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:467)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: io.grpc.StatusRuntimeException: PERMISSION_DENIED: User not authorized to perform this action.
at io.grpc.Status.asRuntimeException(Status.java:526)
... 19 more
The debugger tells me the projectId is set for the correct default project for that service account, so the service account is connected. And, as I said, I verified from the console that the permissions are in fact set to Project:Owner for that service account. So ... why the permission denied?
Thanks in advance for reading gnarly stack traces on my behalf.
Since you are running with no problem with gcloud this must be a problem with IntelliJ. You should check whether you need to start IntelliJ via shell (as described in this article) or set the GOOGLE_APPLICATIONS_CREDENTIALS variable in IntelliJ as described here.
Related
I am attempting to retrieve a secret from aws secrets manager. My application can retrieve 3 / 4 secrets. When attempting to retrieve the 2nd secret, i get an interrupt exception. The code below is what i am using. I am unsure where to look or why this is happening. It consistently happens when attempting to retrieve the 2nd secret. I have verified that it exists in aws and that it contains the value i expect.
I am using the reativerse [https://github.com/reactiverse/aws-sdk] library in tandem with vert.x + aws sdk 2. Any help or pointers on where to look would be greatly appreciated. The exception i see and the code i have are written below.
java.util.concurrent.ExecutionException: software.amazon.awssdk.core.exception.SdkClientException
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:159)
at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:100)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:157)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: software.amazon.awssdk.core.exception.SdkClientException
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
at software.amazon.awssdk.core.internal.http.AmazonAsyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonAsyncHttpClient.java:189)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.invoke(BaseAsyncClientHandler.java:271)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.doExecute(BaseAsyncClientHandler.java:213)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:91)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:277)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:74)
at software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:52)
at software.amazon.awssdk.services.secretsmanager.DefaultSecretsManagerAsyncClient.getSecretValue(DefaultSecretsManagerAsyncClient.java:1142)
... 8 more
Caused by: software.amazon.awssdk.core.exception.SdkInterruptedException
at software.amazon.awssdk.core.internal.http.InterruptMonitor.checkInterrupted(InterruptMonitor.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApplyTransactionIdStage.execute(ApplyTransactionIdStage.java:43)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApplyTransactionIdStage.execute(ApplyTransactionIdStage.java:29)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.AmazonAsyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonAsyncHttpClient.java:185)
... 16 more
private final SecretsManagerAsyncClient client = VertxSdkClient.withVertx(SecretsManagerAsyncClient.builder(), ctx).build();
public <T> Promise<T> getSecret(String secretName) {
Promise<T> completableFuture = Promise.promise();
Context ctx = Vertx.vertx().getOrCreateContext();
ctx.owner().executeBlocking(blockingFuture -> {
try {
GetSecretValueRequest valueRequest = GetSecretValueRequest.builder().secretId(secretName).build();
CompletableFuture<GetSecretValueResponse> valueResponse = this.client.getSecretValue(valueRequest);
GetSecretValueResponse getResponse = valueResponse.get();
String response = getResponse.secretString();
completableFuture.complete((T) Json.decodeValue(response, new <T>Object().getClass()));
blockingFuture.complete();
} catch (Exception e) {
// log exception.
}
});
...
}
See SdkInterruptedException ...it has one method getResponse():
try {
// provoke the exception in here
} catch (SdkInterruptedException e) {
System.err.println(e.getResponse());
}
This should (hopefully) at least tell why the request had been interrupted.
// log exception. is not the correct approach to handle any Exception.
The SdkClientException might be more relevant than the SdkInterruptedException:
try { ...
} catch (SdkClientException e) {
// log that exception, instead of ignoring it.
System.err.println(e.getMessage());
}
This is how an Authorization Header (AWS Signature Version 4) should basically look alike.
It may well be, that HMAC-SHA256 headers are being enforced (they're different on each single request). I could even provide a known to be working example... which would show how to configure the SecretsManagerClient. I'd suspect the root cause related to that client's configuration.
When I read "when attempting to retrieve the 2nd secret", this with high probability means that you need a refresh the HMAC-SHA256 signature before the second request will be performed. Destroying the client and constructing it again may be one sloppy workaround. When seeing duplicate values for AWS4-HMAC-SHA256 header on both requests, this definitely is the cause.
Trying to read avro data from a kafka topic using a flink application. Getting the below error while running the flink app. This is my first time working with flink/kafka, and couldn't fix this from days.
org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.CompletionException: java.lang.IllegalStateException: Failed to rollback to checkpoint/savepoint s3://rinc-ingestion-service/flink-savepoints/namespaces/default/deployments/e8f1afd0-9236-43c2-93cb-db16604da594/1a6716af-6357-4ae3-95da-dad0bdb1f7cc/savepoint-f101ee-feb4e8023b09. Cannot map checkpoint/savepoint state for operator 7df19f87deec5680128845fd9a6ca18d to the new program, because the operator is not available in the new program. If you want to allow to skip this, you can set the --allowNonRestoredState option on the CLI.
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
... 3 more
Caused by: java.lang.IllegalStateException: Failed to rollback to checkpoint/savepoint s3://rinc-ingestion-service/flink-savepoints/namespaces/default/deployments/e8f1afd0-9236-43c2-93cb-db16604da594/1a6716af-6357-4ae3-95da-dad0bdb1f7cc/savepoint-f101ee-feb4e8023b09. Cannot map checkpoint/savepoint state for operator 7df19f87deec5680128845fd9a6ca18d to the new program, because the operator is not available in the new program. If you want to allow to skip this, you can set the --allowNonRestoredState option on the CLI.
at org.apache.flink.runtime.checkpoint.Checkpoints.throwNonRestoredStateException(Checkpoints.java:230)
at org.apache.flink.runtime.checkpoint.Checkpoints.loadAndValidateCheckpoint(Checkpoints.java:194)
at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.restoreSavepoint(CheckpointCoordinator.java:1648)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.tryRestoreExecutionGraphFromSavepoint(DefaultExecutionGraphFactory.java:163)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:138)
at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:335)
at org.apache.flink.runtime.scheduler.SchedulerBase.(SchedulerBase.java:191)
at org.apache.flink.runtime.scheduler.DefaultScheduler.(DefaultScheduler.java:140)
at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:134)
at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:110)
at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:346)
at org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:323)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:106)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:94)
at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
... 3 more
The flink app code is below.
import io.confluent.kafka.serializers.KafkaAvroDeserializerConfig;
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.connector.kafka.source.KafkaSource;
import org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializer;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import testRecord.DataRecordAvro;
public class KafkaAvroDeserialize {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
KafkaSource<DataRecordAvro> source = KafkaSource.<DataRecordAvro>builder()
.setBootstrapServers("pkc-2396y.us-east-1.aws.confluent.cloud:9092")
.setTopics("test")
.setGroupId("demo-consumer-avro-1")
.setProperty(KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, "https://psrc-1wydj.us-east-2.aws.confluent.cloud")
.setProperty("basic.auth.credentials.source", "USER_INFO")
.setProperty("basic.auth.user.info", "5CDO5ZR4HDJSGOHT:v/YJxrw6iR+ASbIwMGefZN9SIzPccuyyTQC8EXFn8cOcJCeC1EASEfvHzFRflyXd")
.setProperty("advertised.host.name", "pkc-2396y.us-east-1.aws.confluent.cloud:9092")
.setValueOnlyDeserializer(new AvroDeserializer<>(DataRecordAvro.class))
.setStartingOffsets(OffsetsInitializer.earliest())
.build();
DataStreamSource<DataRecordAvro> input = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source");
input.map(record -> "DESERIALIZED: " + record.getMsgType() + "-" + record.get(0) + "-" + record.get(1) + "-" + record.get(2)).print();
env.execute("Printing the payload");
}
}
The clue is this entry from the logs:
Caused by: java.lang.IllegalStateException: Failed to rollback to checkpoint/savepoint s3://rinc-ingestion-service/flink-savepoints/namespaces/default/deployments/e8f1afd0-9236-43c2-93cb-db16604da594/1a6716af-6357-4ae3-95da-dad0bdb1f7cc/savepoint-f101ee-feb4e8023b09. Cannot map checkpoint/savepoint state for operator 7df19f87deec5680128845fd9a6ca18d to the new program, because the operator is not available in the new program. If you want to allow to skip this, you can set the --allowNonRestoredState option on the CLI.
You've made changes to the job since it was last run, and as a result there's state in a checkpoint or savepoint that cannot be restored.
You'll want to read this section from the documentation -- https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/state/savepoints/#assigning-operator-ids -- which explains the role played by operator IDs, and the importance of setting explicit IDs so that you can evolve how your job uses state without running into problems like this.
If you don't mind starting over from scratch (and abandoning whatever state your job has), then the easiest way forward is to resubmit the job without having it try to restart from a checkpoint or savepoint. In your case the only state is in the KafkaSource, so hopefully dropping the state won't be painful.
I wasted hours trying the Google Document AI java example from https://cloud.google.com/document-ai/docs/quickstart-client-libraries
If you enter your for projectId, location and processorId like this
String projectId = "6493xxxxxxxx";
String location = "eu";
String processorId = "74451xxxxxx";
String filePath = "/Users/schube/Desktop/file.pdf";
and run the example, you just get an InvalidArgumentException:
Exception in thread "main" com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Request contains an invalid argument.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1133)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1277)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:563)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:557)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:69)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:738)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:717)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.documentai.v1.DocumentProcessorServiceClient.processDocument(DocumentProcessorServiceClient.java:232)
at documentai.QuickStart.quickStart(QuickStart.java:57)
at documentai.QuickStart.main(QuickStart.java:25)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Request contains an invalid argument.
at io.grpc.Status.asRuntimeException(Status.java:535)
... 13 more
That is really great, because which f*** element is invalid? There are no details in the exception and there is no logfile.
So, turns out after reading this: What argument is invalid for Google Document AI client library for Node js?
that if the location is not "us" but "eu", you have to specify an other endpoint.
Oh man, why did they not mention that in the tutorial or the source code? It just says:
// TODO(developer): Replace these variables before running the sample.
String projectId = "your-project-id";
String location = "your-project-location"; // Format is "us" or "eu".
String processorId = "your-processor-id";
String filePath = "path/to/input/file.pdf";
The really should have mentioned that it is not enough to switch to "eu".
So the simple solution is, do not use
try (DocumentProcessorServiceClient client = DocumentProcessorServiceClient.create()) {
but
try (DocumentProcessorServiceClient client = DocumentProcessorServiceClient.create(
DocumentProcessorServiceSettings.newBuilder().setEndpoint("eu-documentai.googleapis.com:443").build())) {
I am trying to create a Google Cloud Function using the Java Client API with this client:
Credentials myCredentials = ServiceAccountCredentials.fromStream(new FileInputStream(keyFile));
CloudFunctionsServiceSettings settings = CloudFunctionsServiceSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(myCredentials)).build();
client = CloudFunctionsServiceClient.create(settings);
String project = "my-project-name";
String location = "us-central1";
LocationName locationName = LocationName.of( project, location );
CloudFunction function = CloudFunction.newBuilder().build();
CloudFunction response = client.createFunctionAsync(locationName, function).get();
I tried different invocations but I'am getting always the following stack trace:
java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: The request has errors
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547)
at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:86)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:62)
at com.google.api.gax.longrunning.OperationFutureImpl.get(OperationFutureImpl.java:127)
at it.myapp.test.App.main(App.java:59)
Caused by: com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: The request has errors
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1041)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1215)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:563)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:464)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:428)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:461)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: The request has errors
at io.grpc.Status.asRuntimeException(Status.java:535)
... 16 more
My pom.xml have the following setup:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-functions-bom</artifactId>
<version>1.0.8</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-functions</artifactId>
</dependency>
</dependencies>
Does anyone know what do i wrong ?
Thank you
The error means that your Function Builder is missing the required parameters in order to create the function. If you try to create a function via Cloud Console, you're required to enter details such as function name, entrypoint, runtime, trigger type, and source code.
I've already reached out to the engineers and they are now informed with regards to the lack of details in the log output.
As a solution, here's a sample code that will create a Cloud Function running on Java 11. Of course you can always choose any type of runtime you want:
package function;
import com.google.cloud.functions.v1.CloudFunctionsServiceClient;
import com.google.cloud.functions.v1.HttpsTrigger;
import com.google.cloud.functions.v1.CloudFunction;
import com.google.cloud.functions.v1.LocationName;
public class App {
public static void main( String[] args ){
try {
// TODO: Add your credentials here
CloudFunctionsServiceClient cloudFunctionsServiceClient = CloudFunctionsServiceClient.create();
String location = LocationName.of("[PROJECT_ID]", "us-central1").toString();
CloudFunction function = CloudFunction.newBuilder()
.setName(location + "/functions/[FUNCTION_NAME]")
.setEntryPoint("functions.HelloHttp") // fully qualified class name (FQN)
.setRuntime("java11")
.setHttpsTrigger(HttpsTrigger.getDefaultInstance())
.setSourceArchiveUrl("gs://[BUCKET_NAME]/source_code.zip")
.build();
CloudFunction response = cloudFunctionsServiceClient.createFunctionAsync(location, function).get();
}catch (Exception e){
e.printStackTrace();
}
}
}
Note: If your zipped source code is from a storage bucket, make sure your source files are at the root of the ZIP file, rather than a folder containing the files.
I'm trying to get started as an AutoML Vision client for the Google Cloud Platform in Java. Simplified the program, it seems to me, to the limit:
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import com.google.cloud.automl.v1.AutoMlClient;
import com.google.cloud.automl.v1.Dataset;
import com.google.cloud.automl.v1.LocationName;
public class Create_DS {
public static void main(String[] args)
throws IOException, ExecutionException, InterruptedException {
try (AutoMlClient autoMlClient = AutoMlClient.create()) {
LocationName parent = LocationName.of("zimin-77", "us-central1");
Dataset dataset = Dataset.newBuilder().build();
Dataset response = autoMlClient.createDatasetAsync(parent.toString(), dataset).get();
System.out.format("LocationName.of() Location name:%s",response.getDescription());
}
}
}
And in any case, when I start, I get a message:
Exception in thread "main" java.util.concurrent.ExecutionException: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 4.944791400s. [buffered_nanos=4948378600, waiting_for_connection]
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:564)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:545)
at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:86)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:62)
at com.google.api.gax.longrunning.OperationFutureImpl.get(OperationFutureImpl.java:127)
at sid_ii.Create_DS.main(Create_DS.java:19)
Caused by: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 4.944791400s. [buffered_nanos=4948378600, waiting_for_connection]
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:51)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1050)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1176)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:969)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:760)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:545)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:515)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
at io.grpc.internal.ClientCallImpl$1CloseInContext.runInContext(ClientCallImpl.java:416)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 4.944791400s. [buffered_nanos=4948378600, waiting_for_connection]
at io.grpc.Status.asRuntimeException(Status.java:533)
... 13 more
I can not do anything !!! Fantasy is over. (((Please help
If I understand correctly, you are having issues when testing the AutoML Vision client on your machine.
The documentation specifies that a DEADLINE_EXCEEDED error message is thrown because the request did not finish within the deadline. In other services the same deadline error can be caused by client runtime, network connectivity, or even firewall.
The AutoML Vision API Tutorial tutorial has Java Code to create the dataset (Step 1: Create the Flowers dataset) with the same method you are using:
client.createDatasetAsync(projectLocation, dataset)
I have followed the guide and it ran correctly in Cloud Shell. In order to discard any issue with the API from your machine or VM instance, try the tutorial code from the Cloud Shell.