Develop Reference

Flink: TaskManager cannot connect to the JobManager - Could not resolve ResourceManager address

I'm using the Apache Flink Kubernetes operator to deploy a standalone job on an Application cluster setup.
I have setup the following files using the Flink official documentation - Link
jobmanager-application-non-ha.yaml
taskmanager-job-deployment.yaml
flink-configuration-configmap.yaml
jobmanager-service.yaml
I have not changed any of the configurations in these files and am trying to run a simple WordCount example from the Flink examples using the Apache Flink Operator.
After running the kubectl commands to setting up the job manager and the task manager - the job manager goes into a NotReady state while the task manager goes into a CrashLoopBackOff loop.
NAME READY STATUS RESTARTS AGE
flink-jobmanager-28k4b 1/2 NotReady 2 (4m24s ago) 16m
flink-kubernetes-operator-6585dddd97-9hjp4 2/2 Running 0 10d
flink-taskmanager-6bb88468d7-ggx8t 1/2 CrashLoopBackOff 9 (2m21s ago) 15m
The job manager logs look like this
org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
at org.apache.flink.runtime.jobmaster.slotpool.PhysicalSlotRequestBulkCheckerImpl.lambda$schedulePendingRequestBulkWithTimestampCheck$0(PhysicalSlotRequestBulkCheckerImpl.java:86) ~[flink-dist-1.16.0.jar:1.16.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:453) ~[flink-rpc-akka_be40712e-8b2e-47cd-baaf-f0149cf2604d.jar:1.16.0]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_be40712e-8b2e-47cd-baaf-f0149cf2604d.jar:1.16.0]
The Task manager it seems cannot connect to the job manager
2023-01-28 19:21:47,647 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Connecting to ResourceManager akka.tcp://flink#flink-jobmanager:6123/user/rpc/resourcemanager_*(00000000000000000000000000000000).
2023-01-28 19:21:57,766 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Could not resolve ResourceManager address akka.tcp://flink#flink-jobmanager:6123/user/rpc/resourcemanager_*, retrying in 10000 ms: Could not connect to rpc endpoint under address akka.tcp://flink#flink-jobmanager:6123/user/rpc/resourcemanager_*.
2023-01-28 19:22:08,036 INFO akka.remote.transport.ProtocolStateActor [] - No response from remote for outbound association. Associate timed out after [20000 ms].
2023-01-28 19:22:08,057 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink#flink-jobmanager:6123] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink#flink-jobmanager:6123]] Caused by: [No response from remote for outbound association. Associate timed out after [20000 ms].]
2023-01-28 19:22:08,069 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Could not resolve ResourceManager address akka.tcp://flink#flink-jobmanager:6123/user/rpc/resourcemanager_*, retrying in 10000 ms: Could not connect to rpc endpoint under address akka.tcp://flink#flink-jobmanager:6123/user/rpc/resourcemanager_*.
2023-01-28 19:22:08,308 WARN akka.remote.transport.netty.NettyTransport [] - Remote connection to [null] failed with org.jboss.netty.channel.ConnectTimeoutException: connection timed out: flink-jobmanager/100.127.18.9:6123
The flink-configuration-configmap.yaml looks like this
flink-conf.yaml: |+
jobmanager.rpc.address: flink-jobmanager
taskmanager.numberOfTaskSlots: 2
blob.server.port: 6124
jobmanager.rpc.port: 6123
taskmanager.rpc.port: 6122
queryable-state.proxy.ports: 6125
jobmanager.memory.process.size: 1600m
taskmanager.memory.process.size: 1728m
parallelism.default: 2
This is what the pom.xml looks like - Link

You deployed the Kubernetes Operator in the namespace, but you did not create the CRDs the Operator requires. Instead you tried to create a standalone Flink Kubernetes cluster.
The Flink Operator makes it a lot easier to deploy your Flink jobs, you only need to deploy the operator itself and FlinkDeployment/FlinkSessionJob CRDs. The operator will manage your deployment after.
Please use this documentation for the Kubernetes Operator: Link

Failed to connect Azure servicebus topic using JMS - Java

I followed steps as mentioned in Azure ServiceBus JMS Sample with below properties
spring.jms.servicebus.connection-string=Endpoint=sb://test-dt.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=key
spring.jms.servicebus.topic-client-id=12345
spring.jms.servicebus.idle-timeout=18000
spring.jms.servicebus.pricing-tier=Standard
however I get below error
ERROR 43904 --- [ntContainer#0-1] org.apache.qpid.jms.JmsConnection : Failed to connect to remote at: amqps://test-dt.servicebus.windows.net:-1
ERROR 43904 --- [ntContainer#0-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'test-topic' - retrying using FixedBackOff{interval=5000, currentAttempts=6, maxAttempts=unlimited}. Cause: handshake timed out after 10000ms
On the other hand, I followed steps as mentioned in ServiceBus without JMS and added transportType as AmqpTransportType.AMQP_WEB_SOCKETS then I am able to connect.
We want to implement using spring boot starter and listener method, instead of calling from (public static void main) method.
Please guide on what am I missing when following first link

ERROR 43904 --- [ntContainer#0-1] org.apache.qpid.jms.JmsConnection : Failed to connect to remote at: amqps://test-dt.servicebus.windows.net:-1
To resolve above error, try as suggested by Anand Sowmithiran:
Check if port 5671 is blocked:
telnet <yournamespacename>.servicebus.windows.net 5671
Note: Clients that use AMQP connections over TCP require ports 5671 and 5672 to be opened in the firewall. Along with these ports, it might be necessary to open additional ports if the EnableLinkRedirect feature is enabled.
You can refer to Troubleshooting guide for Azure Service Bus, AMQP outbound port requirements and Port 5671 Blocked :(. What are other options?

JMS - Couldn't create session factory even if connection to remote provider is OK

I make a jar to connect to JMS provider, it's work fine in a server (a) but doesn't work on a server (b).
I verify that connection to the JMS provider from server (b) is OK via telnet as shown below:
-> telnet -d 10.127.161.247 4447
Trying 10.127.161.247...
Connected to 10.127.161.247.
The logs and exception that it's throw is below :
2018/08/01 11:19:47:921 - Processing command line arguments...
2018/08/01 11:19:47:927 - Preparing the output... Writing to file = true
2018/08/01 11:19:50:315 - Trying to establish connection, attempt: 0
2018/08/01 11:19:52:132 - Connecting to provider: remote://10.127.161.247:4447 using user: oad_user
2018/08/01 11:19:52:132 - Creating a regular (non-secure) connection factory...
2018/08/01 11:19:53:073 - Opening a connection to the JMS provider...
2018/08/01 11:20:24:192 - Attempt failed: Failed to create session factory
javax.jms.JMSException: Failed to create session factory
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:675)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createTopicConnection(ActiveMQConnectionFactory.java:267)
at com.alcatel.ossgw.client.jms.JmsSubscriberClient.prepareConnection(JmsSubscriberClient.java:334)
at com.alcatel.ossgw.client.jms.JmsSubscriberClient.connect(JmsSubscriberClient.java:274)
at com.alcatel.ossgw.client.jms.JmsSubscriberClient.listen(JmsSubscriberClient.java:97)
at com.alcatel.ossgw.client.jms.JmsSubscriberClient.main(JmsSubscriberClient.java:536)
Caused by: ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ119007: Cannot connect to server(s). Tried with all available servers.]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:818)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:67
I don't how to debug such a problem knowing i'm new in JMS, i'll be grateful for any remarks and ideas..
Many thanks

Did you ever find out why this was happening?
I have a related issue where an active JMS connection suddenly disconnects and the system starts throwing this exception until a system restart is done and then it connects again without fail.

Connecting to couchbase running on alternate ports using java client

I'm using couchbase-client 2.3.2 for Java and Couchbase server Community 4.0
So I'm experimenting with running Couchbase on non-standard ports using the documentation at Couchbase website
I've managed to start Couchbase using these alternate ports but I've only managed to change some of the ports in the java client, here's my code:
final CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder()
.bootstrapCarrierDirectPort(21210)
.bootstrapHttpDirectPort(9091)
.build();
return CouchbaseCluster.create(env, "10.0.2.15");
My program is able to connect to couchbase and so some things, however I still need to change the view port (default 8092) and the query port (default 8093) in the client. as a result I'm met with these errors:
2016-09-30 14:03:49.696 [] WARN c.c.c.c.e.Endpoint - [null][QueryEndpoint]: Could not connect to endpoint, retrying with delay 32 MILLISECONDS: ! java.net.ConnectException: Connection refused: /10.0.2.15:8093
2016-09-30 14:03:52.077 [] WARN c.c.c.c.e.Endpoint - [null][ViewEndpoint]: Could not connect to endpoint, retrying with delay 2048 MILLISECONDS: ! java.net.ConnectException: Connection refused: /10.0.2.15:8092
So the client still tries to connect to 8092 and 8093 when in fact I've changed those to 9092 and 9093

From the JavaDoc on 2.3.4 (http://docs.couchbase.com/sdk-api/couchbase-java-client-2.3.4/), I believe what you want is this:
DefaultCouchbaseEnvironment.Builder viewEndpoints(int viewServiceEndpoints)

Even though it's completely undocumented, you need to add those ports to static_config as well:
{capi_port, 9092}.
{query_port, 9093}.
and then it works, hope someone at couchbase sees this and updates their documentation :)

Cassandra-Cli refusing connection

I am trying to connect to cassandra. I installed the latest stable version that is apache-cassandra-1.2.4 and extracted it on my desktop. As I run cassandra it sets up nicely listening for thrift client and displaying following :
sudo cassandra -f
log :
INFO 15:30:34,646 Cassandra version: 1.0.12
INFO 15:30:34,646 Thrift API version: 19.20.0
INFO 15:30:34,646 Loading persisted ring state
INFO 15:30:34,650 Starting up server gossip
INFO 15:30:34,661 Enqueuing flush of Memtable-LocationInfo#1117603949(29/36 serialized/live bytes, 1 ops)
INFO 15:30:34,661 Writing Memtable-LocationInfo#1117603949(29/36 serialized/live bytes, 1 ops)
INFO 15:30:34,877 Completed flushing /var/lib/cassandra/data/system/LocationInfo-hd-54-Data.db (80 bytes)
INFO 15:30:34,892 Starting Messaging Service on port 7000
INFO 15:30:34,901 Using saved token 143186062733850112297005303551620336860
INFO 15:30:34,903 Enqueuing flush of Memtable-LocationInfo#1282534304(53/66 serialized/live bytes, 2 ops)
INFO 15:30:34,904 Writing Memtable-LocationInfo#1282534304(53/66 serialized/live bytes, 2 ops)
INFO 15:30:35,102 Completed flushing /var/lib/cassandra/data/system/LocationInfo-hd-55-Data.db (163 bytes)
INFO 15:30:35,106 Node localhost/127.0.0.1 state jump to normal
INFO 15:30:35,107 Bootstrap/Replace/Move completed! Now serving reads.
INFO 15:30:35,108 Will not load MX4J, mx4j-tools.jar is not in the classpath
INFO 15:30:35,150 Binding thrift service to localhost/127.0.0.1:9160
INFO 15:30:35,155 Using TFastFramedTransport with a max frame size of 15728640 bytes.
INFO 15:30:35,160 Using synchronous/threadpool thrift server on localhost/127.0.0.1 : 9160
INFO 15:30:35,168 Listening for thrift clients...
Now as I run : cassandra-cli -h localhost -p 9160, it throws up the error. I have checked for the port to be free and cassandra is listening at the port. :
**
org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at org.apache.cassandra.cli.CliMain.connect(CliMain.java:80)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:256)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:178)
... 3 more
Exception connecting to localhost/9160. Reason: Connection refused.
**

I had the same error.Now, it is OK.
The main problem is that the configuration is wrong.
My configuration is as following:
My visual machine ip is 192.168.11.11.My cassandra was installed into the machine.So, I configurate thar
listen_address: 192.168.11.11
rpc_address: 0.0.0.0
broadcast_rpc_address: 192.168.11.11
That is OK。

The documentation of cassandra-stress seems to be sketchy. Maybe in due course that would be corrected. As of now, this command worked for me
./cassandra-stress write -node <IP_OF_NODE1>
Once this works, we could try putting in the other optional parameters to tweak our command.

Option 1:
Run jps command under root user and kill CassandraDaemon if you will see it. After this you will start Cassandra again.
Option2:
Try to connect Cassandra with CQL
./cqlsh 10.234.31.232 9042
Final Check:
An intermediate firewall is blocking the JVM from making the connection.
An operating system firewall, or antivirus that is causing the problems as well.
I think you installed in windows and looks like firewall is blocking your connection.