Develop Reference

Infinispan drops connection to MYSQL 5.7 database

We have Keycloak in HA, which we have configured with a external Infinispan cluster for sessions, clientSessions & authenticationSessions.
Everything works under containers in a similar approach like the one performed under https://github.com/albertoSoto/keycloak-infinispan-cluster
The project runs KC 15.0.2 with Wildfly (migration to quarkus to be done), and in that case, uses Infinispan 11.0.9 to perform the external data persistence to Mysql 5.7. The driver used is the latest one, using https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar as suggested by Oracle. The connection driver is com.mysql.cj.jdbc.Driver.
The project starts nice, but after random time, mysql drops the connection and the Infinispan cluster can't reconnect.
In a trial to make it work, I have been able to use Agroal configuration, based in a properties file like it's at the bottom of this message.
The content of that agroal property file, that overrides JPA behavior in the project is the following:
org.infinispan.agroal.metricsEnabled=false
org.infinispan.agroal.minSize=10
org.infinispan.agroal.maxSize=100
org.infinispan.agroal.initialSize=20
org.infinispan.agroal.acquisitionTimeout_s=1
org.infinispan.agroal.validationTimeout_m=1
org.infinispan.agroal.leakTimeout_s=10
org.infinispan.agroal.reapTimeout_m=10
org.infinispan.agroal.maxLifetime_m=10
org.infinispan.agroal.autoCommit=true
org.infinispan.agroal.jdbcTransactionIsolation=READ_COMMITTED
org.infinispan.agroal.jdbcUrl=jdbc:mysql://mysql:3306/infinispan
org.infinispan.agroal.driverClassName=com.mysql.cj.jdbc.Driver
org.infinispan.agroal.principal=keycloak
org.infinispan.agroal.credential=password
The error shown after the connection is closed from the db is the following:
[1;31m21:55:31,052 ERROR (jgroups-319,vi-infinispan-1-5379) [org.infinispan.interceptors.impl.InvocationContextInterceptor] ISPN000136: Error executing command RemoveCommand on Cache 'clientSessions', writing keys [WrappedByteArray{bytes=0304090000000E\j\a\v\a\.\u\t\i\l\.\U\U\I\DBC9903F798\m85\/000000020000000C\l\e\a\s\t\S\i\g\B\i\t\s\$000000000B\m\o\s\t\S\i\g\B\i\t\s\$00168D0C\z8AB49FBA9B\C118A06A0DB\D82... (85 bytes), hashCode=73644551}] org.infinispan.remoting.RemoteException: ISPN000217: Received exception from vi-infinispan-0-53111, see cause for remote stack trace
at org.infinispan.remoting.transport.ResponseCollectors.wrapRemoteException(ResponseCollectors.java:25)
at org.infinispan.remoting.transport.ValidSingleResponseCollector.withException(ValidSingleResponseCollector.java:37)
at org.infinispan.remoting.transport.ValidSingleResponseCollector.addResponse(ValidSingleResponseCollector.java:21)
at org.infinispan.remoting.transport.impl.SingleTargetRequest.addResponse(SingleTargetRequest.java:73)
at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:43)
at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:52)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1402)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1305)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:131)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1445)
at org.jgroups.JChannel.up(JChannel.java:784)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:913)
at org.jgroups.protocols.FRAG3.up(FRAG3.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:876)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049)
at org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:132)
at org.jgroups.protocols.FailureDetection.up(FailureDetection.java:186)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:254)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:281)
at org.jgroups.protocols.Discovery.up(Discovery.java:300)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1396)
at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.infinispan.persistence.spi.PersistenceException: Error while removing string keys from database
at org.infinispan.marshall.exts.ThrowableExternalizer.readObject(ThrowableExternalizer.java:234)
at org.infinispan.marshall.exts.ThrowableExternalizer.readObject(ThrowableExternalizer.java:42)
at org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:728)
at org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:709)
at org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:358)
at org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:32)
at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:49)
at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:41)
at org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:728)
at org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:709)
at org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:358)
at org.infinispan.marshall.core.GlobalMarshaller.objectFromObjectInput(GlobalMarshaller.java:192)
at org.infinispan.marshall.core.GlobalMarshaller.objectFromByteBuffer(GlobalMarshaller.java:221)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1394)
... 25 more
Caused by: java.sql.SQLNonTransientConnectionException: No operations allowed after connection closed.
We do use JDBC_PING for the cluster connection and 2 nodes are active. They register themselves properly and everything works like a charm, until the timeout is set.
The base configuration that I have placed is the following:
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:11.0 https://infinispan.org/schemas/infinispan-config-11.0.xsd
urn:infinispan:server:11.0 https://infinispan.org/schemas/infinispan-server-11.0.xsd"
xmlns="urn:infinispan:config:11.0"
xmlns:server="urn:infinispan:server:11.0">
<!--
Generic XML definition located under
https://docs.jboss.org/infinispan/11.0/configdocs/
-->
<jgroups>
<stack-file name="default-udp" path="default-jgroups.xml"/>
<stack-file name="default-tcp" path="default-jgroups-tcp.xml"/>
<stack-file name="gce" path="default-jgroups-google.xml"/>
<stack-file name="k8s" path="default-jgroups-kubernetes.xml"/>
<stack-file name="kc-udp" path="default-keycloak-jgroups-udp.xml"/>
<stack-file name="custom-k8s-jdbc" path="custom-jgroups-kubernetes-jdbc.xml"/>
<stack-file name="custom-tcp-jdbc" path="custom-jgroups-tcp-jdbc.xml"/>
</jgroups>
<cache-container name="default" statistics="${env.INFINISPAN_CACHE_STATISTICS:false}">
<serialization marshaller="org.infinispan.jboss.marshalling.commons.GenericJBossMarshaller">
<white-list>
<class>java.util.UUID</class>
<regex>org.keycloak.models.sessions.infinispan.*</regex>
</white-list>
</serialization>
<serialization marshaller="org.infinispan.commons.marshall.JavaSerializationMarshaller">
<white-list>
<class>java.util.UUID</class>
<regex>org.keycloak.models.sessions.infinispan.*</regex>
</white-list>
</serialization>
<transport cluster="${infinispan.cluster.name:cluster}" stack="${infinispan.cluster.stack:default-udp}"
node-name="${infinispan.node.name:}"/>
<replicated-cache-configuration name="sessions-cfg" mode="SYNC" start="EAGER"
statistics="${env.INFINISPAN_CACHE_STATISTICS:false}">
<state-transfer timeout="${infinispan.statetransfer.timeout:600000}"/>
<encoding media-type="application/x-jboss-marshalling"/>
<expiration lifespan="900000000000000000"/>
</replicated-cache-configuration>
<distributed-cache-configuration name="distributed-cache-cfg">
<encoding media-type="application/x-jboss-marshalling"/>
<expiration lifespan="900000000000000000"/>
<persistence passivation="false">
<string-keyed-jdbc-store shared="true" xmlns="urn:infinispan:config:store:jdbc:11.0">
<connection-pool properties-file="${env.PROPERTIES_FILE:/opt/infinispan/server/conf/connection-pool.properties}" />
<string-keyed-table drop-on-exit="false"
prefix="ISPN">
<id-column name="ID_COLUMN" type="VARCHAR(255)"/>
<!-- Blob generates error on KC. We increase it to a safe max size (65K per row)
<data-column name="DATA_COLUMN" type="BLOB" />
-->
<data-column name="DATA_COLUMN" type="VARBINARY(50000)"/>
<timestamp-column name="TIMESTAMP_COLUMN" type="BIGINT"/>
<segment-column name="SEGMENT_COLUMN" type="INT"/>
</string-keyed-table>
</string-keyed-jdbc-store>
</persistence>
<state-transfer timeout="${infinispan.statetransfer.timeout:600000}"/>
</distributed-cache-configuration>
<!--https://infinispan.org/docs/stable/titles/configuring/configuring.html#distributed-caches_clustered-caches-->
<!--https://infinispan.org/docs/stable/titles/configuring/configuring.html#configuring-jdbc-cache-stores_persistence-->
<distributed-cache name="sessions" owners="2" configuration="distributed-cache-cfg">
</distributed-cache>
<distributed-cache name="clientSessions" owners="2" configuration="distributed-cache-cfg">
</distributed-cache>
<distributed-cache name="authenticationSessions" owners="2" configuration="distributed-cache-cfg">
</distributed-cache>
</cache-container>
<!-- Original at v11 - -->
<server xmlns="urn:infinispan:server:11.0">
<interfaces>
<interface name="public">
<inet-address value="${infinispan.bind.address:0.0.0.0}"/>
</interface>
</interfaces>
<socket-bindings default-interface="public" port-offset="0">
<socket-binding name="default" port="11222"/>
</socket-bindings>
<security>
<security-realms>
<security-realm name="default">
<properties-realm groups-attribute="Roles">
<user-properties path="users.properties" relative-to="infinispan.server.config.path"
plain-text="true"/>
<group-properties path="groups.properties" relative-to="infinispan.server.config.path"/>
</properties-realm>
</security-realm>
</security-realms>
</security>
<endpoints socket-binding="default" security-realm="default">
<hotrod-connector name="hotrod">
<authentication>
<sasl mechanisms="SCRAM-SHA-512 SCRAM-SHA-384 SCRAM-SHA-256 SCRAM-SHA-1 DIGEST-SHA-512 DIGEST-SHA-384 DIGEST-SHA-256 DIGEST-SHA DIGEST-MD5 PLAIN"
qop="auth" server-name="infinispan"/>
</authentication>
</hotrod-connector>
<rest-connector name="rest">
<authentication mechanisms="DIGEST BASIC"/>
</rest-connector>
</endpoints>
</server>
</infinispan>
The thing is... what are am I doing wrong?
There is not too much information about it. Can anyone help?
Thank you!

Unfortunately I think you have stumbled across a bug with the persistence availability check that prevents stores from reconnecting if an exception is thrown ISPN-13863. I have just created a PR, however the fix will only be available in the Infinispan 14.x stream.

How to set the Apache MQ topic size and time to live and restart active MQ programmatically?

I am new to Apache ActiveMQ. I want to set the number of messages allowed in a topic and the time to live (ttl) of the messages depending on the choice the user makes from the UI. If the user makes a change then restart the ActiveMQ with updated ttl and topic size.
Could someone please let me know how to do this programmatically in Java?
As of now, I have understood that altering the constantPendingMessageLimitStrategy in the activemq.xml file (attached below for reference) by using dom parser in Java will help define the topic size at any given point of time. However, I do not know how to alter "time to live" according to our preference.
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this configuration file -->
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true">
<!-- The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<!--
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
-->
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<!--
The systemUsage controls the maximum amount of space the broker will
use before slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
-->
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!--
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
</beans>

broker.xml not picking up changes

I have followed this tutorial, but I've made the consumer in one Spring Boot app and the producer in another Spring Boot app. The consumer successfully received messages when the producer sends it to the queue.
My queue name is youtube.
What I'm trying to do is set the max-size-bytes to 1 so essentially no messages get received by the consumer listening to the youtube queue. This is what I have in my broker.xml:
<address-setting match="youtube">
<max-size-bytes>10</max-size-bytes>
<page-size-bytes>1</page-size-bytes>
<address-full-policy>DROP</address-full-policy>
</address-setting>
I can see the changes being picked up in the artemis.log, but the consumer is still consuming messages despite the messages being over 10 bytes. How do I properly configure this?
My broker.xml:
<?xml version='1.0'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<configuration xmlns="urn:activemq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xi="http://www.w3.org/2001/XInclude"
xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq:core ">
<name>0.0.0.0</name>
<persistence-enabled>true</persistence-enabled>
<!-- this could be ASYNCIO, MAPPED, NIO
ASYNCIO: Linux Libaio
MAPPED: mmap files
NIO: Plain Java Files
-->
<journal-type>NIO</journal-type>
<paging-directory>data/paging</paging-directory>
<bindings-directory>data/bindings</bindings-directory>
<journal-directory>data/journal</journal-directory>
<large-messages-directory>data/large-messages</large-messages-directory>
<journal-datasync>true</journal-datasync>
<journal-min-files>2</journal-min-files>
<journal-pool-files>10</journal-pool-files>
<journal-device-block-size>4096</journal-device-block-size>
<journal-file-size>10M</journal-file-size>
<!--
This value was determined through a calculation.
Your system could perform 0.14 writes per millisecond
on the current journal configuration.
That translates as a sync write every 7100000 nanoseconds.
Note: If you specify 0 the system will perform writes directly to the disk.
We recommend this to be 0 if you are using journalType=MAPPED and journal-datasync=false.
-->
<journal-buffer-timeout>7100000</journal-buffer-timeout>
<!--
When using ASYNCIO, this will determine the writing queue depth for libaio.
-->
<journal-max-io>1</journal-max-io>
<!--
You can verify the network health of a particular NIC by specifying the <network-check-NIC> element.
<network-check-NIC>theNicName</network-check-NIC>
-->
<!--
Use this to use an HTTP server to validate the network
<network-check-URL-list>http://www.apache.org</network-check-URL-list> -->
<!-- <network-check-period>10000</network-check-period> -->
<!-- <network-check-timeout>1000</network-check-timeout> -->
<!-- this is a comma separated list, no spaces, just DNS or IPs
it should accept IPV6
Warning: Make sure you understand your network topology as this is meant to validate if your network is valid.
Using IPs that could eventually disappear or be partially visible may defeat the purpose.
You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running -->
<!-- <network-check-list>10.0.0.1</network-check-list> -->
<!-- use this to customize the ping used for ipv4 addresses -->
<!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->
<!-- use this to customize the ping used for ipv6 addresses -->
<!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->
<!-- how often we are looking for how many bytes are being used on the disk in ms -->
<disk-scan-period>5000</disk-scan-period>
<!-- once the disk hits this limit the system will block, or close the connection in certain protocols
that won't support flow control. -->
<max-disk-usage>90</max-disk-usage>
<!-- should the broker detect dead locks and other issues -->
<critical-analyzer>true</critical-analyzer>
<critical-analyzer-timeout>120000</critical-analyzer-timeout>
<critical-analyzer-check-period>60000</critical-analyzer-check-period>
<critical-analyzer-policy>HALT</critical-analyzer-policy>
<page-sync-timeout>7100000</page-sync-timeout>
<!-- the system will enter into page mode once you hit this limit.
This is an estimate in bytes of how much the messages are using in memory
The system will use half of the available memory (-Xmx) by default for the global-max-size.
You may specify a different value here if you need to customize it to your needs.
<global-max-size>100Mb</global-max-size>
-->
<acceptors>
<!-- useEpoll means: it will use Netty epoll if you are on a system (Linux) that supports it -->
<!-- amqpCredits: The number of credits sent to AMQP producers -->
<!-- amqpLowCredits: The server will send the # credits specified at amqpCredits at this low mark -->
<!-- amqpDuplicateDetection: If you are not using duplicate detection, set this to false
as duplicate detection requires applicationProperties to be parsed on the server. -->
<!-- amqpMinLargeMessageSize: Determines how many bytes are considered large, so we start using files to hold their data.
default: 102400, -1 would mean to disable large mesasge control -->
<!-- Note: If an acceptor needs to be compatible with HornetQ and/or Artemis 1.x clients add
"anycastPrefix=jms.queue.;multicastPrefix=jms.topic." to the acceptor url.
See https://issues.apache.org/jira/browse/ARTEMIS-1644 for more information. -->
<!-- Acceptor for every supported protocol -->
<acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true</acceptor>
<!-- AMQP Acceptor. Listens on default AMQP port for AMQP traffic.-->
<acceptor name="amqp">tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpMinLargeMessageSize=102400;amqpDuplicateDetection=true</acceptor>
<!-- STOMP Acceptor. -->
<acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>
<!-- HornetQ Compatibility Acceptor. Enables HornetQ Core and STOMP for legacy HornetQ clients. -->
<acceptor name="hornetq">tcp://0.0.0.0:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor>
<!-- MQTT Acceptor -->
<acceptor name="mqtt">tcp://0.0.0.0:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor>
</acceptors>
<security-settings>
<security-setting match="#">
<permission type="createNonDurableQueue" roles="amq"/>
<permission type="deleteNonDurableQueue" roles="amq"/>
<permission type="createDurableQueue" roles="amq"/>
<permission type="deleteDurableQueue" roles="amq"/>
<permission type="createAddress" roles="amq"/>
<permission type="deleteAddress" roles="amq"/>
<permission type="consume" roles="amq"/>
<permission type="browse" roles="amq"/>
<permission type="send" roles="amq"/>
<!-- we need this otherwise ./artemis data imp wouldn't work -->
<permission type="manage" roles="amq"/>
</security-setting>
</security-settings>
<address-settings>
<address-setting match="youtube">
<max-size-bytes>10</max-size-bytes>
<page-size-bytes>1</page-size-bytes>
<address-full-policy>DROP</address-full-policy>
</address-setting>
<!-- if you define auto-create on certain queues, management has to be auto-create -->
<address-setting match="activemq.management#">
<dead-letter-address>DLQ</dead-letter-address>
<expiry-address>ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<!-- with -1 only the global-max-size is in use for limiting -->
<max-size-bytes>-1</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>PAGE</address-full-policy>
<auto-create-queues>true</auto-create-queues>
<auto-create-addresses>true</auto-create-addresses>
<auto-create-jms-queues>true</auto-create-jms-queues>
<auto-create-jms-topics>true</auto-create-jms-topics>
</address-setting>
<!--default for catch all-->
<address-setting match="#">
<dead-letter-address>DLQ</dead-letter-address>
<expiry-address>ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<!-- with -1 only the global-max-size is in use for limiting -->
<max-size-bytes>-1</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>PAGE</address-full-policy>
<auto-create-queues>true</auto-create-queues>
<auto-create-addresses>true</auto-create-addresses>
<auto-create-jms-queues>true</auto-create-jms-queues>
<auto-create-jms-topics>true</auto-create-jms-topics>
</address-setting>
</address-settings>
<addresses>
<address name="DLQ">
<anycast>
<queue name="DLQ" />
</anycast>
</address>
<address name="ExpiryQueue">
<anycast>
<queue name="ExpiryQueue" />
</anycast>
</address>
</addresses>
<!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events
<broker-plugins>
<broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
<property key="LOG_ALL_EVENTS" value="true"/>
<property key="LOG_CONNECTION_EVENTS" value="true"/>
<property key="LOG_SESSION_EVENTS" value="true"/>
<property key="LOG_CONSUMER_EVENTS" value="true"/>
<property key="LOG_DELIVERING_EVENTS" value="true"/>
<property key="LOG_SENDING_EVENTS" value="true"/>
<property key="LOG_INTERNAL_EVENTS" value="true"/>
</broker-plugin>
</broker-plugins>
-->
</core>
</configuration>

JMS/ActiveMQ MessageConsumer.recieve() not returning

I have a Java application that uses what I believe to be fairly boiler plate JMS code to connect to and receive messages from an ActiveMQ server.
this.consumerFactory = new ActiveMQConnectionFactory(this.ingestItemBrokerUrl);
this.consumerConnection = this.consumerFactory.createConnection();
this.consumerConnection.start();
this.consumerSession = this.consumerConnection.createSession(false, Session.AUTO_ACKNOWLEDGE);
this.consumerDestination = this.consumerSession.createTopic(getIngestItemDestinationName());
this.consumer = this.consumerSession.createConsumer(this.consumerDestination);
My application calls MessageConsumer.receive() in a loop (until the connection is closed) to process messages as they arrive in an ActiveMQ topic:
message = this.consumer.receive();
Here's the mystery:
When I connect to an ActiveMQ server running on localhost, this works as expected. But when I connect to an ActiveMQ server running on a azure cloud machine (loaded with Bitnami's ActiveMQ stack), the receive() call blocks indefinitely, even though I can see from the AMQ admin console that my client has connected and dequeued messages.
Why would I be seeing different behavior when switching from local to a remote server? How can I further troubleshoot?
My cloud activemq.xml config file is below:
<beans xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean id="configurationEncryptor" class="org.jasypt.encryption.pbe.StandardPBEStringEncryptor">
<property name="algorithm" value="PBEWithMD5AndDES"/>
<property name="password" value="**REDACTED**"/>
</bean>
<bean id="propertyConfigurer" class="org.jasypt.spring31.properties.EncryptablePropertyPlaceholderConfigurer">
<constructor-arg ref="configurationEncryptor"/>
<property name="location" value="file:${activemq.conf}/credentials-enc.properties"/>
</bean>
<!-- Allows accessing the server log -->
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery" lazy-init="false" scope="singleton" init-method="start" destroy-method="stop">
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<plugins>
<!--simpleAuthenticationPlugin>
<users>
<authenticationUser username="${activemq.username}" password="${activemq.password}" groups="admins"/>
</users>
</simpleAuthenticationPlugin-->
<!-- if not already set, set ttl to 1 minutes -->
<timeStampingBrokerPlugin zeroExpirationOverride="60000"/>
</plugins>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor/>
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<!--
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
-->
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<!--
The systemUsage controls the maximum amount of space the broker will
use before slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
If using ActiveMQ embedded - the following limits could safely be used:
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="20 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="100 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
-->
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="100 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!--
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ssl" uri="ssl://0.0.0.0:61617?maximumConnections=1000&trace=true&needClientAuth=true"/>
</transportConnectors>
<!-- SSL Configuration Context -->
<sslContext>
<sslContext keyStore="file:${activemq.conf}/amq-server.ks"
keyStorePassword="**REDACTED**"
trustStore="file:${activemq.conf}/amq-server.ts"
trustStorePassword="**REDACTED**" />
</sslContext>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
The web consoles requires by default login, you can disable this in the jetty.xml file
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
</beans><!-- END SNIPPET: example -->

Can you try by using this as the destination name this.consumerDestination = this.consumerSession.createTopic(getIngestItemDestinationName() + "?consumer.prefetchSize=1");
or call
consumerFactory.getPrefetchPolicy().setTopicPrefetch(1);
Try with message listener
MessageListener listner = new MessageListener() {
public void onMessage(Message message) {
try {
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
System.out.println("Received message"
+ textMessage.getText() + "'");
}
} catch (JMSException e) {
System.out.println("Caught:" + e);
e.printStackTrace();
}
}
};
consumer.setMessageListener(listner);

Disabling the listening to rabbit queues from spring application.properties

I want to create an application-development.properties file in spring to define a dev environment. In this environment want to disable the listening to the rabbit queues because I don't want to interfere with the staging queues while debugging etc.
Problem is - I can't find a property that controls this. No "active" property or "enabled" property or anything..
These are the properties I found in the Spring docs:
# RABBIT (RabbitProperties)
spring.rabbitmq.addresses= # connection addresses (e.g. myhost:9999,otherhost:1111)
spring.rabbitmq.dynamic=true # create an AmqpAdmin bean
spring.rabbitmq.host= # connection host
spring.rabbitmq.port= # connection port
spring.rabbitmq.password= # login password
spring.rabbitmq.requested-heartbeat= # requested heartbeat timeout, in seconds; zero for none
spring.rabbitmq.listener.acknowledge-mode= # acknowledge mode of container
spring.rabbitmq.listener.concurrency= # minimum number of consumers
spring.rabbitmq.listener.max-concurrency= # maximum number of consumers
spring.rabbitmq.listener.prefetch= # number of messages to be handled in a single request
spring.rabbitmq.listener.transaction-size= # number of messages to be processed in a transaction
spring.rabbitmq.ssl.enabled=false # enable SSL support
spring.rabbitmq.ssl.key-store= # path to the key store that holds the SSL certificate
spring.rabbitmq.ssl.key-store-password= # password used to access the key store
spring.rabbitmq.ssl.trust-store= # trust store that holds SSL certificates
spring.rabbitmq.ssl.trust-store-password= # password used to access the trust store
spring.rabbitmq.username= # login user
spring.rabbitmq.virtual-host= # virtual host to use when connecting to the broker
I did find a way not to load the amqp-context.xml beans that contain the listener definitions by using Spring profiles and add <beans profile="development"> .. </beans> to the xml but this is much less flexible as I have to define different profiles, and changing what they include involves changing the code.
EDIT this is how my amqp-context.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:task="http://www.springframework.org/schema/task"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:rabbit="http://www.springframework.org/schema/rabbit"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/task
http://www.springframework.org/schema/task/spring-task-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/rabbit
http://www.springframework.org/schema/rabbit/spring-rabbit-1.3.xsd">
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="true" />
<property name="locations">
<list>
<value>application.${env:xxxx}.properties</value>
</list>
</property>
</bean>
<rabbit:connection-factory id="connectionFactory" host="${rabbit_host}"
virtual-host="${rabbit_virtual_host}" username="${rabbit_username}" password="${rabbit_password}" port="${rabbit_port}"/>
<!-- Connection Factory -->
<bean id="rabbitConnFactory"
class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
</bean>
<!-- Spring AMQP Template -->
<bean id="template" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
<property name="connectionFactory" ref="connectionFactory" />
<property name="routingKey" value="${my_queue}" />
<property name="queue" value="${my_queue}" />
</bean>
<!-- Spring AMQP Admin -->
<bean id="admin" class="org.springframework.amqp.rabbit.core.RabbitAdmin">
<constructor-arg ref="rabbitConnFactory" />
</bean>
<rabbit:listener-container connection-factory="connectionFactory" requeue-rejected="false" concurrency="10">
<rabbit:listener ref="ProcessMessage"
queue-names="${queue_name}" />
</rabbit:listener-container>
<bean id="ProcessStuff" class="Process" />
</beans>
Does anyone have an idea on how I can manage the listening to queues directly from the application.properties file? please?

As an alternative to waiting for Boot 1.3, you can add your own key to application-development.properties like
rabbit.auto-startup=false
Then modify your amqp-context.xml like this
<rabbit:listener-container connection-factory="connectionFactory" requeue-rejected="false" concurrency="10" auto-startup=${rabbit.auto-startup}>

Good catch! I've created #3587 which will be addressed for Spring Boot 1.3
Thanks!

This one "spring.autoconfigure.exclude: org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration" doesn't help me. So I just remove all properties start with spring.cloud.stream.* and spring.rabbitmq.addresses. Also add to your logback
<logger name="org.springframework.amqp" level="ERROR"/>
<logger name="org.springframework.boot.actuate.amqp" level="ERROR"/>.
Because when you remove the properties, spring output a lot of WARN logs.