I use jersey-client to consume a REST service.
I need both the requested Entity and the Last-Modified header.
So I do the following:
ClientResponse response = webResource.get(ClientResponse.class);
Person person = response.getEntity(Person.class);
That works. I get a response and I can marshal the Entity (wich is XML) into my POJO.
When I debug and take a look into the Headers of the response, then I see that there is a Last-Modified header set.
But when I try to retrieve the date via
response.getLastModified();
I get a NPE somewhere in URLConnectionClientHandler.
Has anyone a clue what I do wrong?
edit: as requested the trace
java.lang.NullPointerException: null
at com.sun.jersey.api.client.ClientResponse.getLastModified(ClientResponse.java:647) ~[jersey-client-1.12.jar:1.12]
at a.o.u.user.dao.impl.uds.PersonenUdsClient.getPerson(PersonenUdsClient.java:103) ~[um-user-2.5.0-Beta1-SNAPSHOT.jar:na]
at a.o.u.user.dao.impl.UserDaoUdsImpl.mergeWithUdsUser(UserDaoUdsImpl.java:282) ~[um-user-2.5.0-Beta1-SNAPSHOT.jar:na]
at a.o.u.user.dao.impl.UserDaoUdsImpl.getUserWithEmail(UserDaoUdsImpl.java:124) ~[um-user-2.5.0-Beta1-SNAPSHOT.jar:na]
at ...
edit: as npe suggested I digged into the code. I think I found the problem. Beside jersey-client I also have cxf in the classpath. Both jersey and cxf provide a class called RuntimeDelegateImpl. But CXFs version does not feature a DateHeaderDelegate. I think the wrong version (CXFs) of RuntimeDelegateImpl is taken.
By now I have not found how I can explicitely set the RuntimeDelegateImpl to use.
Use the source, Luke
The implementation of ClientResponse#getLastModified() for version 1.12 looks like this:
/*639*/ /**
/*640*/ * Get the last modified date.
/*641*/ *
/*642*/ * #return the last modified date, otherwise <code>null</code> if not present.
/*643*/ */
/*644*/ public Date getLastModified() {
/*645*/ String d = getHeaders().getFirst("Last-Modified");
/*646*/
/*647*/ return (d != null) ? dateDelegate.fromString(d) : null;
/*648*/ }
You get a NullPointerException in line 647, so it appears, that dateDelegate is null. Now, the dateDelegate object is initialized in line 321, like this:
/*321*/ protected static final HeaderDelegate<Date> dateDelegate =
/*322*/ RuntimeDelegate.getInstance().createHeaderDelegate(Date.class);
Now, the field is final, so it cannot be changed after this initialization - which means dateDelegate is null from the beginning - and that means, you have some kind of configuration issue and the delegate is not created.
Further, delegates are created in the AbstractRuntimeDelegate class (source for 1.12 here), like this:
/* 88*/ map.put(Date.class, _createHeaderDelegate(Date.class));
This rabbit hole goes deeper and deeper, so I'm going to stop here, but you know the way.
And last, but not least - debugger is your friend, my friend ;-)
To make it a wrap.
The problem is that I have both jersey-client and cxf in the classpath. Both RuntimeDelegateImpl. But CXFs version does not feature a DateHeaderDelegate. The wrong version (CXFs) of RuntimeDelegateImpl is taken.
I solved the issue by retrieving the Last-Modified Header 'manually':
private Date getLastModified(ClientResponse response){
String lastModifiedString = response.getHeaders().getFirst(
"Last-Modified");
if (StringUtils.isEmpty(lastModifiedString)) {
LOG.warn("Expect to get Last-Modified header when retrieving a Person by pnr "
+ "but there is none.");
return null;
} else {
try {
// format is Thu, 21 Jun 2012 08:00:42 GMT
return new SimpleDateFormat(
"EEE, dd MMM yyyy HH:mm:ss zzz", Locale.US)
.parse(lastModifiedString);
} catch (ParseException e) {
LOG.error("Could not parse Last-Modified date "
+ e.getMessage());
return null;
}
}
}
Thanx to npe for the hints.
Related
W've just upgraded our application from Hazelcast 3.8.0 to 3.10.1.
We am getting an error message "key cannot be of type Data!" when accessing data in Hazelcast.
java.lang.IllegalArgumentException: key cannot be of type Data!
at com.hazelcast.util.Preconditions.checkNotInstanceOf(Preconditions.java:300)
at com.hazelcast.internal.nearcache.impl.DefaultNearCache.checkKeyFormat(DefaultNearCache.java:226)
at com.hazelcast.internal.nearcache.impl.DefaultNearCache.get(DefaultNearCache.java:114)
at com.hazelcast.map.impl.tx.TransactionalMapProxySupport.getCachedValue(TransactionalMapProxySupport.java:183)
at com.hazelcast.map.impl.tx.TransactionalMapProxySupport.getInternal(TransactionalMapProxySupport.java:132)
at com.hazelcast.map.impl.tx.TransactionalMapProxy.get(TransactionalMapProxy.java:110)
at com.hazelcast.client.impl.protocol.task.transactionalmap.TransactionalMapGetMessageTask.innerCall(TransactionalMapGetMessageTask.java:43)
at com.hazelcast.client.impl.protocol.task.AbstractTransactionalMessageTask.call(AbstractTransactionalMessageTask.java:34)
at com.hazelcast.client.impl.protocol.task.AbstractCallableMessageTask.processMessage(AbstractCallableMessageTask.java:35)
at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:130)
at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:110)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
at ------ submitted from ------.(Unknown Source)
I am 100% sure that the "key" We're using is a String. The code snippet looks like this:
String key = getKey(blah, blah);
TransactionContext context = client.newTransactionContext();
context.beginTransaction();
TransactionalMap<String, AppPrefs> dataMap = context.getMap(MAP_NAME);
try {
prefs = dataMap.get(key);
context.commitTransaction();
} catch (Throwable t) {
LOGGER.error("Error getting AppPrefs.", t);
context.rollbackTransaction();
}
The line of code throwing the error is:
prefs = dataMap.get(key);
There is nothing between the line of code that sets the String
String key = getKey(blah, blah);
and the line that blows :(
Following the source code through, the TransactionalMapProxySupport's method "toNearCacheKeyWithStrategy" has this logic:
final Object toNearCacheKeyWithStrategy(Object key) {
if (!nearCacheEnabled) {
return key;
}
return serializeKeys ? ss.toData(key, partitionStrategy) : key;
}
The "DefaultNearCache" object then does a
private void checkKeyFormat(K key) {
if (!serializeKeys) {
checkNotInstanceOf(Data.class, key, "key cannot be of type Data!");
}
}
So it looks like the property of "TransactionalMapProxySupport" called "serializeKeyes" must have been FALSE but the same-named property in "DefaultNearCache"must have been true :(
The history of "DefaultNearCache" in git shows that the code was changed a year ago with the message: "Changed serialize-keys default for Near Cache from true to false"
Is there some configuration I should be setting?
The config I have for the map is simply:
<near-cache name="AppPrefsCache">
<!-- Cache locally for 10 mins -->
<max-idle-seconds>600</max-idle-seconds>
</near-cache>
Ah! It appears that I can add a " serialize-keys" tag of (true|false) to this xml and set the value.
It appears that different bits of the Hazelcast codebase are assuming a different default value for "serialize-keys". Either it should be a mandatory element in the config or the default should be the same everywhere, yes?
The mentioned behavior is really a bug in Hazelcast. I've created a new GitHub issue for it, so we can handle it properly - hazelcast/hazelcast#13371.
The workaround for versions affected by the bug is to set serialize-keys to true in the near-cache configuration. The false value won't help here.
Config config = new Config();
config.getMapConfig(testName)
.setNearCacheConfig(new NearCacheConfig().setSerializeKeys(true));
Thanks for the report and great investigation, Peter.
I'm doing an integration testing with DBUnit (2.49) + Hibernate (4.1.3) following this tutorial.
Production database : Oracle 10
Test database : Hsqldb 2.3.3
Context
My data contains the current format of date : yyyy/MM/dd. However,according to DBUnit faq, DBUnit only supports this format yyyy-mm-dd hh:mm:ss.fffffffff, so I had to create a new format for TimeStamp.
How I tried to fix it
I created a CustomTimeStampDataType based on this tutorial. I changed this part:
String formats[] = {"yyyy-MM-dd HH:mm", "yyyy-MM-dd HH:mm a", "yyyy-MM-dd HH:mm:ss.fffffffff"};
into this one:
String formats[] = {"yyyy/MM/dd"};
I created a CustomeDataTypeFactory following the same tutorial. I only make it extend Oracle10DataTypeFactory rather than DefaultDatatTypeFactory.
In HibernateDBUnitTestCase, I override setDatabaseConfig() with the following:
#Override
protected void setUpDatabaseConfig(DatabaseConfig config){
config.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new CustomDataTypeFactory());
}
But I got new errors
I ran a unit test and got this error.
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997/02/14> of type <java.lang.String> to TIMESTAMP
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:120)
at org.dbunit.dataset.datatype.TimestampDataType.setSqlValue(TimestampDataType.java:176)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
Caused by: java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
at java.sql.Timestamp.valueOf(Unknown Source)
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:116)
... 20 more
That was weird, it seemed like my CustomTimeStamp was not called, so I changed the date in the dataset using the default format : 1997-02-14 00:00:00.0, and ran the unit test again. Then I got:
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997-02-14 00:00:00.0> of type <java.lang.String> to TIMESTAMP
at test.CustomTimestampDataType.typeCast(CustomTimestampDataType.java:69)
at test.CustomTimestampDataType.setSqlValue(CustomTimestampDataType.java:84)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
That means CustomTimeStamp was actually called. Seems like, the problem stemed from DatabaseTestCase.setUp which somehow called the wrong TimeStampDataType.
How could I fix this issue?
My first option was to replace every yyyy/MM/dd into yyyy-mm-dd in the dataset using regular expressions. This worked fine, until I had to test a method that selected a date based on a request (so the format is yyyy-mm-dd) and compared it to the current date. ( so the format is yyyy / mm / dd). Hsqldb can't compare two dates with different format.
My second option was to decompile dbunit.jar, rewrite TimeStampDataType based on the tutorial. I'm unfamiliar with bytecode writing so before entering uncharted waters, I wanted to know if you had another solution.
Thank you in advance
Fixed it!
So I ended up using my second option.
This is the detailed path for those who need it.
Download dbUnit.2.2.source.jar
Unzip the jar
Go to Eclipse, File > New > Java Project
Uncheck "Use default location"
In Location : specify the path to the new folder created from the jar
Click on Finish
Modify the TimestampDataType.java (if needed)
Instead of ts = java.sql.Timestamp.valueOf(stringValue); use the code below
String formats[] =
{"dd/MM/yyyy HH:mm:ss.SS"}; //and more depending on your need
Timestamp ts = null;
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
ts = new Timestamp(date.getTime());
return ts;
}
catch( ParseException e) {
}
Modify the DateDataType.java (if needed)
Instead of return java.sql.Date.valueOf(stringValue); , use the code below
String formats[] =
{"dd/MM/yyyy"}; //and more depending on your need
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
java.sql.Date datesql = new java.sql.Date(date.getTime());
return datesql;
}
catch( ParseException e) {
}
}
Right-click on your project, then Export
Select JAR file, then Next
Fill the export destination then Finish.
You just have to add this new jar to the library to make it work.
How do you add days to a date in SmartGwt. I found this question and found that I can use CalendarUtil.addDaysToDate(dateToAddTo, daysToAddToDateAsInteger)); but addDaysToDate() is static void. What is the point of a method that can "add days to a date" if it does not return anything?
How do I use this method? I want to do something like this.
Date newDate = dateToAddTo + daysToAddToDate;
Here is a simplified version of my code.
if (listGridRecord.getAttribute("expirationDays") != null) {
CalendarUtil.addDaysToDate((Date) endDate.getValue(), Integer.parseInt(listGridRecord.getAttribute("expirationDays")));
listGridRecord.setAttribute("expirationDate", (Date) endDate.getValue());
} else {
listGridRecord.setAttributeAsJavaObject("expirationDate", null);
}
Here is a link to the javadocs
This method changes the object that is passed as parameter.
Date date = new Date();
long checkBeforeChange = date.getTime();
CalendarUtil.addDaysToDate(date, 1);
long checkAfterChange = date.getTime();
if(checkBeforeChange != checkAfterChange)
Window.alert("It works ;)");
Your code should be something like that:
if (listGridRecord.getAttribute("expirationDays") != null) {
Date tmpDate = endDate.getValue();
CalendarUtil.addDaysToDate(tmpDate, Integer.parseInt(listGridRecord.getAttribute("expirationDays")));
listGridRecord.setAttribute("expirationDate", tmpDate);
} else {
listGridRecord.setAttributeAsJavaObject("expirationDate", null);
}
When doing (Date) endDate.getValue() you get a copy of Date object thus you don't see any change.
I figured out what I was doing wrong with the help of #Adam. I created a new Date variable called expireationDate and set it to (Date) endDate.getValue(); after this I used it to do the calculation.
if (listGridRecord.getAttribute("expirationDays") != null) {
Date expirationDate = (Date) endDate.getValue();
CalendarUtil.addDaysToDate(expirationDate, Integer.parseInt(listGridRecord.getAttribute("expirationDays")));
listGridRecord.setAttribute("expirationDate", expirationDate);
} else {
listGridRecord.setAttributeAsJavaObject("expirationDate", null);
}
First of all, you can wrap all those utility methods you need in your own utility class (private constructor), where e.g. MyDateUtilsClassName.addDays(Date, int) will return new instance, leave parameter unmodified.
When it comes to Date manipulation, in Gwt you can use standard java.util.Date methods, even those deprecated ones like setMinutes, setHours etc. Even when you see com.google.gwt.user.datepicker.client.CalendarUtil method, they are used there.
If it's not server side, but client side, you should not care much about #Deprecated on those methods. Gwt compiles them to javascript anyway. You should be aware of java.util.Calendar. As far as I remember, it is not supported at all.
I am having trouble fixing a Log Forging issue in Fortify. The issue, "writes unvalidated user input to the log", is being raised from both of the logging calls in the getLongFromTimestamp() method.
public long getLongFromTimestamp(final String value) {
LOGGER.info("getLongFromTimestamp(" + cleanLogString(value) + ")");
long longVal = 0;
Date tempDate = null;
try {
tempDate = new SimpleDateFormat(FORMAT_YYYYMMDDHHMMSS, Locale.US).parse(value);
} catch (ParseException e) {
LOGGER.warn("Failed to convert to Date: " + cleanLogString(value) + " Exception: " + cleanLogString(e.getMessage()));
throw new Exception(e);
}
if (tempDate != null) {
longVal = tempDate.getTime();
}
return longVal;
}
private cleanLogString(String logString) {
String clean = logString.replaceAll("[^A-Za-z0-9]", "");
if(!logString.equals(clean)) {
clean += " (CLEANED)";
}
return clean;
}
The cleanLogString() method has fixed other Log Forging Fortify issues in my project, however it has no effect on the 2 above.
Any help would be appreciated!
It is possible to use fortify Java annotations to tell Fortify that the data returned from a sanitizing function is now safe.
When looking at my log forging problems I had strings coming in through a web API and thus had the flags XSS and WEB on my strings. I tried to find annotations that would only remove these flags, but couldn't find any way to remove the WEB flag. The only documentation I've found is the Samples/advanced/javaAnnotation directory.
Since my sanitation method does sanitize strings, I choose to remove all flags. This could be a problem though, as it could hide privacy violations.
#FortifyValidate("return")
private String sanitizeString(String taintedString) {
return doSomethingWithTheString(taintedString);
}
Originally when this question was written our team was using log4j v1.2.8, however we noticed that all the log forging issues disappeared after upgrading to log4j v2.6.2.
Once the log4j version is upgraded the Fortify log forging issues should go away. The cleanLogString() method form the question above is also unnecessary. For example:
LOGGER.info("getLongFromTimestamp(" + value + ")");
I know I have run into situations where the complexity of my application would stop any malicious input from working as intended; Fortify does not consider this to be secure. I bet you are running into the same thing.
You are stripping any really useful characters out of the log message, but see what happens if you do some encoding on the output prior to writing to the log.
http://www.jtmelton.com/2010/09/21/preventing-log-forging-in-java/
// ensure no CRLF injection into logs for forging records
String clean = message.replace( '\n', '_' ).replace( '\r', '_' );
if ( ESAPI.securityConfiguration().getLogEncodingRequired() ) {
clean = ESAPI.encoder().encodeForHTML(message);
if (!message.equals(clean)) {
clean += " (Encoded)";
}
}
Use reflect or try-catch.
Its easy to cheat fortify.
Need a quick help. I am a newbie in QuickFixJ. I have a FIX message in a txt file. I need to convert that into FIX50SP2 format. I am enclosing the code snippet.
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
System.out.println("FixMsg String:"+fixMsg);
Message FIXMessage = new Message();
DataDictionary dd = new DataDictionary("FIX50SP2.xml");
FIXMessage.fromString(fixMsg, dd, false);
System.out.println("FIXMessage Output:" + FIXMessage.toString()); // Print message after parsing
MsgType msgType = new MsgType();
System.out.println(FIXMessage.getField(msgType));
Here is the output:
FixMsg String:1128=99=15835=X49=CME34=47164052=2012090312102051175=20120904268=1279=122=848=336683=607745107=ESU2269=1270=140575271=123273=121020000336=2346=501023=110=205
FIXMessage Output:9=6135=X34=47164049=CME52=2012090312102051175=20120904268=110=117
quickfix.FieldNotFound: Field [35] was not found in message.
at quickfix.FieldMap.getField(FieldMap.java:216)
at quickfix.FieldMap.getFieldInternal(FieldMap.java:353)
at quickfix.FieldMap.getField(FieldMap.java:349)
at MainApp.main(MainApp.java:52)
I want to extract MsgType field (field 35). Could you please tell me where I am wrong? The thing I have observed is that after parsing to FIX50SP2 format, the convert FIX message is missing many data element (for details see the output)
Thanks
Like others mentioned the MsgType is an header field and you get it by using the following
String msgType = null;
if(FIXMessage.getHeader().isSetField(MsgType.FIELD)) {
msgType = FIXMessage.getHeader().getString(MsgType.FIELD);
}
System.out.println("MsgType is " + msgType);`
The reason you are missing many data element after parsing is, probably your message have some custom tags(like tag 2346), which is not defined in your data dictionary(FIXSP02.xml). hence the parsing of those tags failed and missing in the output.
To fix this, get the data dictionary from the party that is sending you the message and use it to parse the message
I'm not familiar with FIX messages and QuickFixJ, but glancing at the Javadoc, it seems like you should use the identifyType method :
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
MsgType msgType = Message.identifyType(fixMsg);
You may find FixB framework useful as it deals well with non-standard use cases of FIX.
As in your case, to extract only data you are interested in, you need to define a class that will represent this data and to bind it to FIX using annotations. E.g.:
#FixBlock
public class MDEntry {
#FixField(tag=269) public int entryType; // you could define an enum type for it as well
#FixField(tag=278) public String entryId;
#FixField(tag=55) public String symbol;
}
...
FixFieldExtractor fixExtractor = new NativeFixFieldExtractor();
List<MDEntry> mdEntries = fixExtractor.getGroups(fixMsg, List.class, 268, FixMetaScanner.scanClass(MDEntry.class))
In more common cases, FixSerializer interface should be used, but it requires a message with MsgType(35) tag and a class annotated with #FixMessage(type="...") accordingly. E.g.:
#FixMessage(type="X")
public class MarketData {
#FixGroup(tag=268) public List<MDEntry> entries;
}
...
FixMetaDictionary fixMetaDictionary = FixMetaScanner.scanClassesIn("my.fix.classes.package");
FixSerializer fixSerializer = new NativeFixSerializer("FIX.5.0.SP2", fixMetaDictionary);
MarketData marketData = fixSerializer.deserialize(fixMsg);
I hope you will find it useful.
If you need just a MsgTyp, you're sure the message is correct and you do not need any other field from the message, then I would recommend extracting MsgType from string using regexp.
e.g.: \u000135=(\w+)\u0001
It is MUCH FASTER than parsing (and validating) a string via QuickFix.