I am writing a Java batch which is listening messages from a queue (Oracle AQ) and sending them to another queue (Tibco EMS), where they are processed by a Tibco process (BW).
The problem is that the Oracle AQ driver automatically add some properties to messages (JMSXGroupID, JMSXGroupSeq) which cause errors when they are processed by Tibco process because they have bad values : JMSXGroupSeq should be an int but is set to null. Tibco falls into error when trying to parse message properties...
So I would like to remove only these 2 properties from all messages but it seems that the jms api only offers a clearProperties() method but no single property remove method (I'm using ths javax.jms.Message interface).
For now, I can see two solutions :
set a correct value to these 2 properties, as I'm assuming they will
not be used further by Tibco
read all properties and reconstruct the messages without the 2 which cause problem. But this approach is very ugly...
Does anyone have any other solution?
It is not possible to edit/clear some properties. We need to call clearProperties method as described here to get write access :
Once a message is produced (sent), its properties become read-only; the properties cannot be changed. While consumers can read the properties using the property accessor methods (getProperty( )), they cannot modify the properties using any of the mutator methods (setProperty( )). If the consumer attempts to set a property, the mutator method throws a javax.jms.MessageNotWriteableException.
Once a message is received, the only way its properties can be changed is by clearing out all the properties using the clearProperties( ) method. This removes all the properties from the message so that new ones can be added. Individual properties cannot be modified or removed once a message is sent.
There will be a function public void removeProperty(String name) in the concrete class implementation of javax.jms.Message interface. This classs is provider specific(Tibco EMS in your case). As it is closed source I cannot be for sure about the existence of that function. But it is present in HornetQ.It can be used to reset particular header property.
Other than that I thing option 1 is the best. You set it to some non null value acceptable by Message header parser of Tibco EMS.
Related
For a Spring application I want to add a custom field to the log.
Currently I use the default format but I want to add a custom field (category field) which should be present in all the logs:
W:Action D:2022-01-10 23:21:03.285 L:INFO C:c.h.l.LoggingDemoApplication F:StartupInfoLogger.java(61) Fn:logStarted T:main R: - Hello World
What are the best solution to add a custom field to the logback log?
What I studied until now are the following possible solutions:
Use marker. The disadvantage with this is that it's not scalable: if in future you need another custom field can't add another marker. Further based on some other posts the marker is best suited to mark special logs that need to be handle differently.
Use MDC.
Also using this it seems not the best solution because:
It keeps the context so if there are multiple log statements in the same function, before each logger.info() there should be MDC.put("category", "action")
The code becomes to verbose.
Create a custom convertor (link). Get the arguments from the ILoggingEvent, get argument of 0. If this is the same type as category enum, then use it. The call for this is like logger.info("Message here: {} {} {}", CatEnum.Action.getValue(), msg1, msg2, msg3).
Create some static method in which the final format is generated.
Pattern is similar to: <pattern>%m%n</pattern>
To log, something like this should be used: logger.info(customFormatter.fmtLog(CatEnum.Action.getValue(), msg)). The returned value of fmtLog should be all the info from default logging + the category field.
Are there any built in solutions to add a custom field?
I have a configuration which if enables blocks unknown variables from passing through.
#Value("${json.failOnUnknown:false}")
private boolean failOnUnknown;
Jackson2ObjectMapperBuilder build = new Jackson2ObjectMapperBuilder();
if(!failOnUnknown) {
build.featuresToDisable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
}
else {
build.featuresToEnable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
}
I want this so if someone sends a bad property to my service I block them. However, my service connects to other services and if they send in an unknown variable it fails as well. I want unknown variables to be ignored when my other services talk to my current service.
I have tried using
#JsonIgnoreProperties(ignoreUnknown=true)
To overwrite the FAIL_ON_UNKNOWN_PROPERTIES but it doesn't work.
Any ideas on how to block unknown variables in some classes and not others?
One solution you might get into is to create two separate ObjectMappers, one that ignores unknown properties and the other one that throws exceptions. You can disable fail on unknow property directly on your Object Mapper scope as following: objectMapper.disable(FAIL_ON_UNKNOWN_PROPERTIES);.
The idea is that you could still create a global scoped object mapper as shown in your example you use almost everywhere except for those services that needs to fail on unknown properties whatever the context is.
I would like to use placeholders in a feature file, like this:
Feature: Talk to two servers
Scenario: Forward data from Server A to Server B
Given MongoDb collection "${db1}/foo" contains the following record:
"""
{"key": "value"}
"""
When I send GET "${server1}/data"
When I forward the respone to PUT "${server2}/data"
Then MongoDB collection "${db2}/bar" MUST contain the following record:
"""
{"key": "value"}
"""
The values of ${server1} etc. would depend on the environment in which the test is to be executed (dev, uat, stage, or prod). Therefore, Scenario Outlines are not applicable in this situation.
Is there any standard way of doing this? Ideally there would be something which maintains a Map<String, String> that can be filled in a #Before or so, and runs automatically between Cucumber and the Step Definition so that inside the step definitions no code is needed.
Given the following step definitions
public class MyStepdefs {
#When("^I send GET "(.*)"$)
public void performGET(final String url) {
// …
}
}
And an appropriate setup, when performGET() is called, the placeholder ${server1} in String uri should already be replaced with a lookup of a value in a Map.
Is there a standard way or feature of Cucumber-Java of doing this? I do not mind if this involves dependency injection. If dependency injection is involved, I would prefer Spring, as Spring is already in use for other reasons in my use case.
The simple answer is that you can't.
The solution to your problem is to remove the incidental details from your scenario all together and access specific server information in the step defintions.
The server and database obviously belong together so lets describe them as a single entity, a service.
The details about the rest calls doesn't really help to convey what you're
actually doing. Features don't describe implementation details, they describe behavior.
Testing if records have been inserted into the database is another bad practice and again doesn't describe behavior. You should be able to replace that by an other API call that fetches the data or some other process that proves the other server has received the information. If there are no such means to extract the data available you should create them. If they can't be created you can wonder if the information even needs to be stored (your service would then appear to have the same properties as a black hole :) ).
I would resolve this all by rewriting the story such that:
Feature: Talk to two services
Scenario: Forward foobar data from Service A to Service B
Given "Service A" has key-value information
When I forward the foobar data from "Service A" to "Service B"
Then "Service B" has received the key-value information
Now that we have two entities Service A and Service B you can create a ServiceInformationService to look up information about Service A and B. You can inject this ServiceInformationService into your step definitions.
So when ever you need some information about Service A, you do
Service a = serviceInformationService.lookup("A");
String apiHost = a.getApiHost():
String dbHost = a.getDatabaseHOst():
In the implementation of the Service you look up the property for that service System.getProperty(serviceName + "_" + apiHostKey) and you make sure that your CI sets A_APIHOST and A_DBHOST, B_APIHOST, B_DBHOST, ect.
You can put the name of the collections in a property file that you look up in a similar way as you'd look up the system properties. Though I would avoid direct interaction with the DB if possible.
The feature you are looking for is supported in gherkin with qaf. It supports to use properties defined in properties file using ${prop.key}. In addition it offers strong resource configuration features to work with different environments. It also supports web-services
I have a class that defines 2 JMSListeners. 1 listener is for a message MyMessage1 that the client receives from a server that uses JMS. Another is for a MyMessage2, which is generated by another program, not using JMS (not a java program). It seems when I try to receive this message, I get some JMS Listener exceptions. It seems to be taking a GenericMessage and attempting to convert it to a MyMessage, despite the payload indicating it's a MyMessage2, although it doesn't specify the full java package name for it.
What exactly is a GenericMessage? Is this an indication that the sender is not properly identifying the message's type? Do I need to define a JMSListener for a generic message somehow and serialize it manually, rather than using the JmsListenerContainerFactory?
#JmsListener()
private void consumeMessage1(final MyMessge1 msg) {
//...
}
#JmsListener()
private void consumeMessage2(final MyMessge2 msg) {
//...
}
org.springframework.jms.listener.adapter.ListenerExecutionFailedException: Listener method could not be invoked with incoming message
Endpoint handler details:
Method [private void my.proj.ResultQueueConsumer.consumeMessage1(my.proj.MyMessage1)]
Bean [my.proj.ResultQueueConsumer#24fe9d1]
; nested exception is org.springframework.messaging.converter.MessageConversionException: No converter found to convert to class my.proj.MyMessage1, message=GenericMessage [payload=MyMessage2{attr1='response', version='2.0', attr2='PROJ', attr3='ERROR'}, headers={JMS_AMQP_FirstAcquirer=false, jms_timestamp=0, hostId=my-machine1, requestId=c83b5719-daaa-43f8-9444-b16580d06950, jms_priority=4, jms_messageId=ID:my-machine1-41383-1486390497702-3:127:0:0:1, timestamp=1486391400112, id=33e78ff8-9e7b-1436-ee21-36cacb1ff6b5, JMS_AMQP_MESSAGE_FORMAT=0, jms_deliveryMode=1, jms_redelivered=false, JMS_AMQP_NATIVE=false, jms_destination=queue://Result, sessionId=057907c44eeb18e3940278973a610b2638d198c4, jms_expiration=0}]
A GenericMessage<> is a spring-messaging object.
Incoming JMS messages are first converted to a GenericMessage and then the listener is invoked with components therefrom; for example, you could have a method signature
void foo(#Payload MyMessage2 msh, #Header("foo") int someHeader)
The error message indicates that a message with a MyMessage2 has been received and the framework is trying to invoke your fist listener method.
If they are both listening to the same queue, the message will randomly go to one or the other listener, unless you add a selector and can somehow determine what the payload type is from a header (used in the selector expression).
The framework does not provide a mechanism to route to a particular method based on the payload type.
It's not clear what you mean by...
Another is for a MyMessage2, which is generated by another program, not using JMS (not a java program).
... since we clearly were able to build a MyMessage2 java object from the message.
Can anyone suggest any design pattern to dynamically differentiate the memcahce instances in java code?
Previously in my application there is only one memcache instance configured this way
Step-1:
dev.memcached.location=33.10.77.88:11211
dev.memcached.poolsize=5
Step-2:
Then i am accessing that memcache in code as follows,
private MemcachedInterface() throws IOException {
String location =stringParam("memcached.location", "33.10.77.88:11211");
MemcachedClientBuilder builder = new XMemcachedClientBuilder(AddrUtil.getAddresses(location));
}
Then i am invoking that memcache as follows in code using above MemcachedInterface(),
Step-3:
MemcachedInterface.getSoleInstance();
And then i am using that MemcachedInterface() to get/set data as follows,
MemcachedInterface.set(MEMCACHED_CUSTS, "{}");
resp = MemcachedInterface.gets(MEMCACHED_CUSTS);
My question is if i introduce an new memcache instance in our architechture,configuration is done as follows,
Step-1:
dev.memcached.location=33.10.77.89:11211
dev.memcached.poolsize=5
So, first memcache instance is in 33.10.77.88:11211 and second memcache instance is in 33.10.77.89:11211
until this its ok...but....
how to handle Step-2 and Step-3 in this case,To get the MemcachedInterface dynamically.
1)should i use one more interface called MemcachedInterface2() in step-2
Now the actual problem comes in,
I am having 4 webservers in my application.Previoulsy all are writing to MemcachedInterface(),but now as i will introduce one more memcache instance ex:MemcachedInterface2() ws1 and ws2 should write in MemcachedInterface() and ws3 and ws4 should write in ex:MemcachedInterface2()
So,if i use one more interface called MemcachedInterface2() as mentioned above,
This an code burden as i should change all the classes using WS3 and WS4 to Ex:MemcachedInterface2() .
Can anyone suggest one approach with limited code changes??
xmemcached supports constistent hashing which will allow your client to choose the right memcached server instance from the pool. You can refer to this answer for a bit more detail Do client need to worry about multiple memcache servers?
So, if I understood correctly, you'll have to
use only one memcached client in all your webapps
since you have your own wrapper class around the memcached client MemcachedInterface, you'll have to add some method to this interface, that enables to add/remove server to an existing client. See the user guide (scroll down a little): https://code.google.com/p/xmemcached/wiki/User_Guide#JMX_Support
as far as i can see is, you have duplicate code running on different machines as like parallel web services. thus, i recommend this to differentiate each;
Use Singleton Facade service for wrapping your memcached client. (I think you are already doing this)
Use Encapsulation. Encapsulate your memcached client for de-couple from your code. interface L2Cache
For each server, give them a name in global variable. Assign those values via JVM or your own configuration files via jar or whatever. JVM: --Dcom.projectname.servername=server-1
Use this global variable as a parameter, configure your Service getInstance method.
public static L2Cache getCache() {
if (System.getProperty("com.projectname.servername").equals("server-1"))
return new L2CacheImpl(SERVER_1_L2_REACHIBILITY_ADDRESSES, POOL_SIZE);
}
good luck with your design!
You should list all memcached server instances as space separated in your config.
e.g.
33.10.77.88:11211 33.10.77.89:11211
So, in your code (Step2):
private MemcachedInterface() throws IOException
{
String location =stringParam("memcached.location", "33.10.77.88:11211 33.10.77.89:11211");
MemcachedClientBuilder builder = new XMemcachedClientBuilder(AddrUtil.getAddresses(location));
}
Then in Step3 you don't need to change anything...e.g. MemcachedInterface.getSoleInstance(); .
You can read more in memcached tutorial article:
Use Memcached for Java enterprise performance, Part 1: Architecture and setup
http://www.javaworld.com/javaworld/jw-04-2012/120418-memcached-for-java-enterprise-performance.html
Use Memcached for Java enterprise performance, Part 2: Database-driven web apps
http://www.javaworld.com/javaworld/jw-05-2012/120515-memcached-for-java-enterprise-performance-2.html