I started to use the pubsub emulator to test my basic implementations and ran into an issue while trying to create a new topic.
My emulator listens on localhost:8085 and if i create the topic via the api
PUT http://localhost:8085/v1/projects/testproject/topics/test
everything works fine and the topic gets created.
But if i run the following snippet nothing works as intended and no topic gets created:
TopicName topicName = TopicName.create("testproject", "test");
ChannelProvider channelProvider =
TopicAdminSettings.defaultChannelProviderBuilder()
.setEndpoint("localhost:8085")
.setCredentialsProvider(
FixedCredentialsProvider.create(NoCredentials.getInstance()))
.build();
TopicAdminClient topicClient = TopicAdminClient.create(
TopicAdminSettings.defaultBuilder().setChannelProvider(channelProvider).build());
topicClient.createTopic(topicName);
while running this the emulator logs
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFORMATION: Adding handler(s) to newly registered Channel.
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFORMATION: Detected non-HTTP/2 connection.
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.netty.NotFoundHandler handleRequest
[pubsub] INFORMATION: Unknown request URI: /bad-request
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFORMATION: Adding handler(s) to newly registered Channel.
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFORMATION: Detected non-HTTP/2 connection.
[pubsub] Apr 27, 2017 1:10:47 PM io.gapi.emulators.netty.NotFoundHandler handleRequest
[pubsub] INFORMATION: Unknown request URI: /bad-request
...
[pubsub] Apr 27, 2017 1:10:49 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFORMATION: Adding handler(s) to newly registered Channel.
[pubsub] Apr 27, 2017 1:10:49 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFORMATION: Detected non-HTTP/2 connection.
Am i missing something on my ChannelProvider? Or didn't I configure my TopicAdminClient correctly? I don't see whats wrong since i used
this as reference.
Maybe someone can help me out with this.
Channels used to communicate with the emulator need to set the negotiationType property to NegotiationType.PLAINTEXT. That means you need to create a custom ChannelProvider. Something like the following should work:
public class PlainTextChannelProvider implements ChannelProvider {
#Override
public boolean shouldAutoClose() {
return false;
}
#Override
public boolean needsExecutor() {
return false;
}
#Override
public ManagedChannel getChannel() throws IOException {
return NettyChannelBuilder.forAddress("localhost", 8085)
.negotiationType(NegotiationType.PLAINTEXT)
.build();
}
#Override
public ManagedChannel getChannel(Executor executor) throws IOException {
return getChannel();
}
}
This post is a little old, hope this serves as an update.
The code snippet from Testing apps locally with the emulator also works. The full snippet is on GitHub if you follow the "View on GitHub" link on the linked page.
String hostport = System.getenv("PUBSUB_EMULATOR_HOST");
ManagedChannel channel = ManagedChannelBuilder.forTarget(hostport).usePlaintext().build();
try {
TransportChannelProvider channelProvider =
FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel));
CredentialsProvider credentialsProvider = NoCredentialsProvider.create();
TopicAdminClient topicClient =
TopicAdminClient.create(
TopicAdminSettings.newBuilder()
.setTransportChannelProvider(channelProvider)
.setCredentialsProvider(credentialsProvider)
.build());
try {
response = topicClient.createTopic(topicName);
System.out.printf("Topic %s created.\n", response);
} catch (ApiException e) {
System.out.println(e.getStatusCode().getCode());
System.out.println(e.isRetryable());
System.out.println("No topic was created.");
}
} finally {
channel.shutdown();
}
Related
I've setup Pubsub.Builder with root url pointed to local PubSub emulator (localhost:8036 at my case). It seems to be working, I see that emulator is receiving my test pushes, but gives me error 400 Bad Request.
Publish code:
auth = GoogleCredential.getApplicationDefault(HTTP_TRANSPORT, JSON_FACTORY)
.createScoped(PubsubScopes.all());
client = new Pubsub.Builder(HTTP_TRANSPORT, JSON_FACTORY, auth)
.setApplicationName "test"
.setRootUrl("http://localhost:8036/")
.build();
msg = new PubsubMessage().encodeData("{\"a\": 1, \"b\": 2}".getBytes());
req = new PublishRequest().setMessages(Arrays.asList(msg));
client.projects()
.topics()
.publish("projects/GCLOUD-DEFAULT-PROJECT/topics/test-topic", req)
.execute();
In emulator console output I see following (that's for very basic PublishRequest):
[pubsub] jan 27, 2016 9:03:20 PM com.google.cloud.pubsub.testing.v1.FakePubsubGrpcServer$2 operationComplete
[pubsub] INFO: Adding handler(s) to newly registered Channel.
[pubsub] jan 27, 2016 9:03:20 PM com.google.cloud.pubsub.testing.v1.NettyUtil$HttpVersionDecider channelRead
[pubsub] INFO: Detected non-HTTP/2 connection.
[pubsub] jan 27, 2016 9:03:20 PM com.google.cloud.pubsub.testing.v1.NettyUtil$HttpJsonAdapter channelRead
[pubsub] INFO: Invalid input: Expect message object but got: "\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�M-.NLO-V���VJI"
[pubsub] com.google.protobuf.InvalidProtocolBufferException: Expect message object but got: "\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�V�M-.NLO-V���VJI"
[pubsub] at com.google.protobuf.util.JsonFormat$ParserImpl.mergeMessage(JsonFormat.java:1099)
[pubsub] at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:1075)
[pubsub] at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:973)
[pubsub] at com.google.protobuf.util.JsonFormat$Parser.merge(JsonFormat.java:201)
[pubsub] at com.google.cloud.pubsub.testing.v1.PubsubJsonGrpcAdapters$PublisherAdapter.handleRequest(PubsubJsonGrpcAdapters.java:231)
[pubsub] at com.google.cloud.pubsub.testing.v1.NettyUtil$HttpJsonAdapter.channelRead(NettyUtil.java:94)
[pubsub] at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
Seems that default PubSub client uses different protocol, but I don't see any way to configure it.
How can I use com.google.apis:google-api-services-pubsub library with local emulator?
The emulator does not require or handle authentication or authorization; I'm guessing that's where the problem lies. Can you try passing null as the last parameter to the builder?
Also, it appears that Gzip compression must be explicitly disabled. Please set this on the builder:
.setGoogleClientRequestInitializer(new GoogleClientRequestInitializer() {
#Override
public void initialize(AbstractGoogleClientRequest<?> request) throws IOException {
request.setDisableGZipContent(true);
}
})
I am trying to connect to an Elasticsearch cluster form a Java application by using the TransportClient.
I have successfully tested connected to a cluster running locally.
Now we have set up a remote cluster in the EC2-Cloud. Accessing it via REST intefaces work, also Kibana works.
I can run my application on the Elasticsearch Node directly successfully.
However when I try to connect remotely from another host, the log indicates that the connection succeeds, but then immediately fails with "An established connection was aborted by the software in your host machine".
I have verified that both server and client run the same version of Elasticsearch. Also the firewall configuration should allow communication on port 9300 as this is also needed for the cluster-nodes to communicate between each other.
Okt 21, 2015 6:50:24 PM com.example.elasticsearch.visit.nativescript.ElasticsearchTest main
INFO: Setting adress to myhost.com:9300
Okt 21, 2015 6:50:24 PM org.elasticsearch.client.transport.TransportClientNodesService addTransportAddresses
FINE: [Master Man] adding address [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]]
Okt 21, 2015 6:50:24 PM org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler doSample
FINEST: [Master Man] connecting to listed node (light) [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]]
Okt 21, 2015 6:50:24 PM org.elasticsearch.transport.netty.NettyTransport connectToNode
FINE: [Master Man] connected to node [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]]
Okt 21, 2015 6:50:24 PM org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker addWithoutBreaking
FINEST: [Master Man] [REQUEST] Adjusted breaker by [16440] bytes, now [16440]
Okt 21, 2015 6:50:24 PM org.elasticsearch.transport.TransportService$Adapter traceRequestSent
FINEST: [Master Man] [0][cluster:monitor/nodes/info] sent to [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]] (timeout: [5s])
Okt 21, 2015 6:50:24 PM org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker addWithoutBreaking
FINEST: [Master Man] [REQUEST] Adjusted breaker by [-16440] bytes, now [0]
Okt 21, 2015 6:50:24 PM org.elasticsearch.transport.netty.NettyTransport exceptionCaught
FINEST: [Master Man] close connection exception caught on transport layer [[id: 0xd938d867, /x.x.x.x:60058 => myhost.com/1.1.1.1:9300]], disconnecting from relevant node
java.io.IOException: An established connection was aborted by the software in your host machine
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Okt 21, 2015 6:50:24 PM org.elasticsearch.transport.netty.NettyTransport disconnectFromNode
FINE: [Master Man] disconnecting from [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]], channel closed event
Okt 21, 2015 6:50:24 PM org.elasticsearch.transport.netty.NettyTransport disconnectFromNode
FINEST: [Master Man] disconnected from [[#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]]], channel closed event
Okt 21, 2015 6:50:24 PM org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler doSample
INFO: [Master Man] failed to get node info for [#transport#-1][LNZ123675N02][inet[myhost.com/1.1.1.1:9300]], disconnecting...
org.elasticsearch.transport.NodeDisconnectedException: [][inet[myhost.com/1.1.1.1:9300]][cluster:monitor/nodes/info] disconnected
Okt 21, 2015 6:50:24 PM com.example.elasticsearch.visit.nativescript.ElasticsearchTest main
INFO: Having: []
Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200)
at org.elasticsearch.client.transport.support.InternalTransportClient.execute(InternalTransportClient.java:106)
at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:167)
at org.elasticsearch.client.transport.TransportClient.bulk(TransportClient.java:370)
at org.elasticsearch.action.bulk.BulkRequestBuilder.doExecute(BulkRequestBuilder.java:166)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at com.example.elasticsearch.visit.nativescript.ElasticsearchTest.main(ElasticsearchTest.java:68)
My simple test looks like this:
Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", "cluster-name")
.build();
logger.info("Creating client");
try (final TransportClient client = new TransportClient(settings)) {
logger.info("Setting adress to " + host + ":" + port);
client.addTransportAddress(new InetSocketTransportAddress(host, port));
logger.info("Having: " + client.connectedNodes());
BulkRequestBuilder bulkRequest = client.prepareBulk();
String json= "{ id: \"12345678\", value: \"value\" }";
bulkRequest.add(client.prepareIndex(INDEX, TYPE, "12345678").
setSource(json));
BulkResponse bulkResponse = bulkRequest.execute().actionGet();
if (bulkResponse.hasFailures()) {
logger.info("Could not write bulk: " + bulkResponse.buildFailureMessage());
} else {
logger.info("Wrote");
}
}
I also tried to verify some common things like cluster-name not set and others.
Also when I use a completely different port, I get a timeout as expected, not this error-message. So the port is definitely behaving different.
Where would I start to look? Can this still be a firewall issue where connections are dropped only after a short while instead of being rejected immediately?
With more testing I actually found out that a company-firewall was interfering in a strange way. When I run the same client-code on a machine outside the company-netowrk, it works fine.
I have written a JAXMPPClient.java program which uses tigase.jaxmpp.j2se.Jaxmpp 3.0.0
I have the following .jar files in my build path
jaxmpp-core-3.0.0
jaxmpp-j2se-3.0.0
and folder jaxmpp-all-3.0.0 containing many .jar files. The JAXMPPClient.java is as follows
Jaxmpp conn = new Jaxmpp();
conn.getConnectionConfiguration().setConnectionType(
ConnectionType.socket);
conn.getConnectionConfiguration().setDomain(IP_ADDRESS_OF_XMPP_SERVER);
conn.getConnectionConfiguration().setPort(PORT_NUMBER_OF_XMPP_SERVER);
conn.getConnectionConfiguration().setUserJID(FROM_USERID + "#" +
IP_ADDRESS_OF_XMPP_SERVER);
conn.getConnectionConfiguration().setUserPassword(FROM_PASSWORD);
String s = tigase.jaxmpp.j2se.connectors.socket.
SocketConnector.HOSTNAME_VERIFIER_DISABLED_KEY
conn.getSessionObject().setProperty(s, Boolean.TRUE);
try {
System.out.println("Before login");
conn.login();
System.out.println("After login");
} catch (final JaxmppException e) {
e.printStackTrace();
}
I am getting
Before login
Sep 29, 2015 5:17:12 PM tigase.jaxmpp.j2se.Jaxmpp createConnector
INFO: Using SocketConnector
Sep 29, 2015 5:17:12 PM tigase.jaxmpp.j2se.connectors.socket.SocketConnector start
INFO: Resolving SRV recrd of domain '123.45.678.910'
Sep 29, 2015 5:17:12 PM tigase.jaxmpp.j2se.connectors.socket.SocketConnector start
INFO: Opening connection to /123.45.678.910:5222
Sep 29, 2015 5:17:12 PM tigase.jaxmpp.j2se.connectors.socket.Worker run
WARNING: Exception in worker
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at tigase.jaxmpp.j2se.connectors.socket.TextStreamReader.read(TextStreamReader.java:49)
at tigase.jaxmpp.j2se.connectors.socket.Worker.run(Worker.java:99)
After login
The IP Address, Port number, From JabberID and Password are correct and are working fine from iOS and Android Apps. Also I am able to successfully ping the Tigase server IP address.
From where is the java.net.SocketException being thrown ? How can I fix this problem ?
How do I turn off the logging in my Console output? I don't want to see the output of my Java program:
Nov 01, 2013 12:01:29 PM org.glassfish.tyrus.server.ServerContainerFactory create
INFO: Provider class loaded: org.glassfish.tyrus.container.grizzly.GrizzlyEngine
Nov 01, 2013 12:01:29 PM org.glassfish.grizzly.http.server.NetworkListener start
INFO: Started listener bound to [0.0.0.0:8029]
Nov 01, 2013 12:01:29 PM org.glassfish.grizzly.http.server.HttpServer start
INFO: [HttpServer] Started.
Nov 01, 2013 12:01:29 PM org.glassfish.tyrus.server.Server start
INFO: WebSocket Registered apps: URLs all start with ws://localhost:8029
Nov 01, 2013 12:01:29 PM org.glassfish.tyrus.server.Server start
INFO: WebSocket server started.
I am surprised that Java does not have this functionality baked in (or that I couldn't find it which is kind of the same thing nowadays)...
Assign the System.out and System.err to a stream that does not write anything (to log). In .NET there is the the Stream.Null which is a stream with no backing store, i.e. a null stream or a void stream.
In Java, such a Stream is easily built by extending the java.io.OutputStream:
class VoidStream extends OutputStream {
#Override
public void write(int b) throws IOException {
//
// --------------
// | void space |
// --------------
//
}
}
Now, set the VoidStream to the System.out and System.err using System.setOut() and System.Err() resulting in an empty console,
PrintStream ps = new PrintStream(new VoidStream());
System.setOut(ps);
System.setErr(ps);
In theory, one caveat is System.console(), becuase its output stream cannot be set but functions as printf() and format() can be called. Fortunately, the System.console() is only available when the program is run in a terminal (otherwise it returns null) and who is still working with a terminal? ;-)
I'm running the following code that errors when I try to commit my changes using Cayenne as my ORM. The code is pasted below and errors out on the context.commitChanges();line. The output messages are pasted below the code. Any help on figuring this out would be appreciated.
import org.apache.cayenne.access.DataContext;
import java.util.*;
import com.jared.*;
public class Main {
public static void main(String[] args) {
DataContext context = DataContext.createDataContext();
Stocks theStock=(Stocks) context.createAndRegisterNewObject(Stocks.class);
theStock.setAsk(3.4);
theStock.setAvgdailyvolume(323849);
theStock .setBid(5.29);
theStock.setChange(-1.22);
theStock.setDayhigh(9.21);
theStock.setDaylow(2.11);
theStock.setLasttradeprice(5.11);
theStock.setLasttradesize(3827);
theStock.setOpen(6.21);
theStock.setPriorclose(4.21);
theStock.setShortratio(1.1);
theStock.setSymbol("^SP%");
theStock.setVolume(28193);
theStock.setLasttradedate(new Date());
context.commitChanges();
System.out.println("Done with the database");
}
}
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate startedLoading
INFO: started configuration loading.
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate shouldLoadDataDomain
INFO: loaded domain: stocks
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate loadDataMap
INFO: loaded .
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate shouldLoadDataNode
INFO: loading .
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate shouldLoadDataNode
INFO: using factory: org.apache.cayenne.conf.DriverDataSourceFactory
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.DriverDataSourceFactory load
INFO: loading driver information from 'stocksNode.driver.xml'.
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.DriverDataSourceFactory$DriverHandler init
INFO: loading driver org.hsqldb.jdbcDriver
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.DriverDataSourceFactory$LoginHandler init
INFO: loading user name and password.
Nov 20, 2008 11:20:37 PM org.apache.cayenne.access.QueryLogger logPoolCreated
INFO: Created connection pool: jdbc:hsqldb:file:/hsqldb/data/stocks
Driver class: org.hsqldb.jdbcDriver
Min. connections in the pool: 1
Max. connections in the pool: 1
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate shouldLoadDataNode
INFO: loaded datasource.
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate initAdapter
INFO: no adapter set, using automatic adapter.
Nov 20, 2008 11:20:37 PM org.apache.cayenne.conf.RuntimeLoadDelegate finishedLoading
INFO: finished configuration loading in 203 ms.
Exception in thread "main" org.apache.cayenne.CayenneRuntimeException: [v.3.0M4 May 18 2008 16:32:02] Commit Exception
at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:1192)
at org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:1066)
at Main.main(Main.java:24)
Caused by: java.lang.NullPointerException
at org.apache.cayenne.access.DataDomainInsertBucket.createPermIds(DataDomainInsertBucket.java:101)
at org.apache.cayenne.access.DataDomainInsertBucket.appendQueriesInternal(DataDomainInsertBucket.java:76)
at org.apache.cayenne.access.DataDomainSyncBucket.appendQueries(DataDomainSyncBucket.java:80)
at org.apache.cayenne.access.DataDomainFlushAction.preprocess(DataDomainFlushAction.java:183)
at org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:135)
at org.apache.cayenne.access.DataDomain.onSyncFlush(DataDomain.java:821)
at org.apache.cayenne.access.DataDomain$2.transform(DataDomain.java:788)
at org.apache.cayenne.access.DataDomain.runInTransaction(DataDomain.java:847)
at org.apache.cayenne.access.DataDomain.onSync(DataDomain.java:785)
at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:1164)
... 2 more
user name, password?
ClientConnection connection = new
HessianConnection("http://localhost:8080/cayenne-service",
"cayenne-user", "secret",
null);