Having a very simple poc such as this one:
IndexedChronicle chronicle = getChronicle("basic");
ExcerptAppender appender = chronicle.createAppender();
appender.startExcerpt();
appender.writeObject(new MessageKey("type", 123L));
appender.finish();
ExcerptTailer tailer = chronicle.createTailer();
while(tailer.nextIndex()) {
MessageKey key = (MessageKey) tailer.readObject();
System.out.println("key " + key);
}
VanillaChronicle vcron = getVainllaChronicle("vanilla");
VanillaAppender app = vcron.createAppender();
app.startExcerpt();
app.writeObject(new MessageKey("type", 123L));
app.finish();
ExcerptTailer vtail = vcron.createTailer();
while(vtail.nextIndex()) {
MessageKey key = (MessageKey) vtail.readObject();
System.out.println("key " + key);
}
Gives me an IndexOutOfBoundsException in the writeObject method on the VanillaAppender.
However, there is little difference, and nothing exceptionally different in the docs
Can anyone suggest how it should be used?
Update:
I re-arranged the code so it became identical to peters (copied it in, actually), but I still get this exception:
Exception in thread "main" java.lang.IndexOutOfBoundsException: position is beyond the end of the buffer 372 > -190495716
at net.openhft.lang.io.NativeBytes.checkEndOfBuffer(NativeBytes.java:518)
at net.openhft.lang.io.AbstractBytes.writeObject(AbstractBytes.java:1897)
at main.ChronicleTest.main(ChronicleTest.java:31)
The version imported is 3.2.1
<dependency>
<groupId>net.openhft</groupId>
<artifactId>chronicle</artifactId>
<version>3.2.1</version>
</dependency>
When I try this with Chronicle 3.2.1
public class SO25623856Main {
public static void main(String[] args) throws IOException {
Chronicle vcron = new VanillaChronicle("vanilla");
ExcerptAppender app = vcron.createAppender();
app.startExcerpt();
app.writeObject(new MessageKey("type", 123L));
app.finish();
ExcerptTailer vtail = vcron.createTailer();
while (vtail.nextIndex()) {
MessageKey key = (MessageKey) vtail.readObject();
System.out.println("key " + key);
}
vcron.close();
}
}
class MessageKey implements Serializable {
private String type;
private long l;
public MessageKey(String type, long l) {
this.type = type;
this.l = l;
}
#Override
public String toString() {
return "MessageKey{" +
"type='" + type + '\'' +
", l=" + l +
'}';
}
}
it prints
key MessageKey{type='type', l=123}
BTW I suggest you use Externalizable or ByteMarshallable for improved performance and smaller messages.
Related
So I'm trying to get my head around using Jackson Annotations, and i'm making requests against Riot's API. This is the response I'm getting: http://jsonblob.com/568079c8e4b01190df45d254. Where the array after the summonerId (38584682) can be of a varying length.
The unique summoner ID will be different every single time too.
I want to map this response to a DTO.
For a similar situation with a different call I am doing:
#JsonIgnore
protected Map<String, SingleSummonerBasicDTO> nonMappedAttributes;
#JsonAnyGetter
public Map<String, SingleSummonerBasicDTO> getNonMappedAttributes() {
return nonMappedAttributes;
}
#JsonAnySetter
public void setNonMappedAttributes(String key, SingleSummonerBasicDTO value) {
if (nonMappedAttributes == null) {
nonMappedAttributes = new HashMap<String, SingleSummonerBasicDTO>();
}
if (key != null) {
if (value != null) {
nonMappedAttributes.put(key, value);
} else {
nonMappedAttributes.remove(key);
}
}
}
From an answer on here. my thinking is to do a for-each loop for each of the elements in the array, but I don't know how to loop over something without having something to loop over.
I am completely stuck as to how the annotations work and how to proceed, any help if appreciated!
First of all, #JsonAnySetter was meant to deal with the case of varying properties, not for json arrays of varying length.
Jackson is quite capable of using Java Collections and Maps in serialization and deserialization. You just have to tell it the parameter type of the collection.
In your case, I have used a Map to capture the root element, making it the sole key with a List of DTOs as value. I use Jackson's type system (TypeFactory and JavaType) to tell jackson of all generic types.
This is the DTO that I have used:
public class SingleSummonerBasicDTO
{
public String name;
public String tier;
public String queue;
public List<SingleSummonerBasicDTOEntry> entries;
#Override
public String toString() {
String toString = "\nSingleSummonerBasicDTO: " + name + " " + tier + " " + queue;
for (SingleSummonerBasicDTOEntry entry : entries) {
toString += "\n" + entry.toString();
}
return toString;
}
public static class SingleSummonerBasicDTOEntry
{
public String playerOrTeamId;
public String playerOrTeamName;
public String division;
public int leaguePoints;
public int wins;
public int losses;
public boolean isHotStreak;
public boolean isVeteran;
public boolean isFreshBlood;
public boolean isInactive;
#Override
public String toString() {
return "Entry: " + playerOrTeamId + " " + playerOrTeamName + " " + division + " " + leaguePoints + " " + wins + " " +
losses + " " + isHotStreak + " " + isVeteran + " " + isInactive;
}
}
this is how to deserialise:
public static void main(String[] args)
{
ObjectMapper mapper = new ObjectMapper();
TypeFactory factory = mapper.getTypeFactory();
// type of key of response map
JavaType stringType = factory.constructType(String.class);
// type of value of response map
JavaType listOfDtosType = factory.constructCollectionLikeType(ArrayList.class, SingleSummonerBasicDTO.class);
// create type of map
JavaType responseType = factory.constructMapLikeType(HashMap.class, stringType, listOfDtosType);
try (InputStream is = new FileInputStream("C://Temp/xx.json")) {
Map<String, List<SingleSummonerBasicDTO>> response = new ObjectMapper().readValue(is, responseType);
System.out.println(response);
} catch (IOException e) {
e.printStackTrace();
}
}
output:
{38584682=[
SingleSummonerBasicDTO: Viktor's Masterminds PLATINUM RANKED_SOLO_5x5
Entry: 38584682 Lazrkiller V 64 291 295 false true false,
SingleSummonerBasicDTO: Renekton's Horde SILVER RANKED_TEAM_5x5
Entry: TEAM-ff7d0db0-78ca-11e4-b402-c81f66dba0e7 Y U NO BABAR II 0 4 2 false false false,
SingleSummonerBasicDTO: Pantheon's Chosen SILVER RANKED_TEAM_5x5
Entry: TEAM-d32018f0-d998-11e4-bfd2-c81f66dba0e7 Lose and Throw Away I 66 7 0 false false false,
SingleSummonerBasicDTO: Jayce's Duelists SILVER RANKED_TEAM_5x5
Entry: TEAM-6c8fc440-a8ac-11e4-b65b-c81f66db920c TopBlokesNeverToke III 0 20 18 false false false]}
Here is the way how to parse your json:
Map<String, List<SingleSummonerBasicDTO>> summonersMap = new ObjectMapper()
.readValue(json, new TypeReference<HashMap<String, List<SingleSummonerBasicDTO>>>() {});
I'm new to AOP, I've created an aspect to trace all methods or classes marked with #Trace annotation. I'm using compile time weaving. (Java 8, Aspectj 1.8, Spring 4)
TraceAspect.java
#Aspect
public class TraceAspect {
private static Map<String, Integer> threadMap = new HashMap<>();
#Pointcut("#within(Trace) || #annotation(Trace)")
void annotated(){}
#Around("annotated() && execution(* *(..))")
public Object trace(final ProceedingJoinPoint joinPoint) throws Throwable {
String threadName = Thread.currentThread().getName();
String indent = indent(inThread(threadName));
System.out.println(threadName + " : " + indent + "-> " + joinPoint.getSignature().toString());
long start = System.nanoTime();
Object ret = joinPoint.proceed();
long end = System.nanoTime();
System.out.println(threadName + " : " + indent + "<- " + joinPoint.getSignature().toString() + " ended (took " + (end - start) + " nanoseconds)");
outThread(threadName);
return ret;
}
private String indent(int depth) {
String result = "";
for (int index = 0; index < depth; index++) {
result += " ";
}
return result;
}
private int inThread(String threadName) {
if (threadMap.get(threadName) == null) {
threadMap.put(threadName, 0);
}
int stackDepth = threadMap.get(threadName) + 1;
threadMap.put(threadName, stackDepth);
return stackDepth;
}
private void outThread(String threadName) {
int stackDepth = threadMap.get(threadName) - 1;
threadMap.put(threadName, stackDepth);
}
}
The CryptsyExchange.java (which is a Spring Bean) when marked with #Trace, classloader throws ClassFormat error on build(..) method while initializing that bean in application context...
CryptsyExchange.java
#Trace
public class CryptsyExchange {
private static final Logger LOGGER = LoggerFactory.getLogger(CryptsyExchange.class);
private DataService dataService;
private Configuration config;
private Converter converter;
private Exchange exchange;
private List<CryptsyAccount> accounts = Collections.synchronizedList(new LinkedList<>());
private CryptsyAccount defaultAccount;
public static CryptsyExchange build(String name, DataService dataService, ConfigPathBuilder pathBuilder) {
condition(notNullOrEmpty(name) && notNull(dataService, pathBuilder));
CryptsyExchange cryptsyExchange = new CryptsyExchange();
cryptsyExchange.dataService = dataService;
// Loading configuration
final Configuration configuration = Configuration.load(pathBuilder.getExchangeConfigPath(name));
cryptsyExchange.config = configuration;
// Retrieve corresponding exchange from datastore
cryptsyExchange.exchange = dataService.registerExchange(cryptsyExchange.config.getString("exchange"));
// Get accounts from configuration
Map<String, Map<String, String>> accounts = configuration.getMap("accounts");
// Initialize accounts
accounts.entrySet().stream().forEach((entry) -> {
String key = entry.getKey();
Map<String, String> accountMap = entry.getValue();
// Retrieve corresponding datastore account
Account account = dataService.registerAccount(cryptsyExchange.exchange, key);
// Initialize cryptsy specific account
CryptsyAccount cryptsyAccount = new CryptsyAccount(account, accountMap.get("key"), accountMap.get("secret"));
cryptsyExchange.accounts.add(cryptsyAccount);
if (notNull(accountMap.get("isDefault")) && Boolean.valueOf(accountMap.get("isDefault"))) {
cryptsyExchange.defaultAccount = cryptsyAccount;
}
});
// Initializing Converter
cryptsyExchange.converter = cryptsyExchange.new Converter();
// Recover associations from configuration
Map<String, String> exchangeCurrencyToCurrency = configuration.getMap("exchangeCurrencyToCurrency");
Set<String> markedForRemoval = new HashSet<>();
exchangeCurrencyToCurrency.entrySet().stream().forEach((entry) -> {
String cryptsyCurrencyCode = entry.getKey();
String currencySymbol = entry.getValue();
com.jarvis.data.entity.Currency currency = dataService.getCurrency(currencySymbol);
if (notNull(currency)) {
cryptsyExchange.converter.associateCurrency(currency, cryptsyCurrencyCode);
} else {
LOGGER.debug("associated currency [" + currencySymbol + "] does not exist in database, removing from configuration");
markedForRemoval.add(cryptsyCurrencyCode);
}
});
// Removing currency associations missing from database
if (!markedForRemoval.isEmpty()) {
markedForRemoval.forEach((currency) -> configuration.remove("exchangeCurrencyToCurrency", currency));
}
Map<String, String> exchangeMarketToMarket = configuration.getMap("exchangeMarketToMarket");
markedForRemoval.clear();
exchangeMarketToMarket.entrySet().stream().forEach((entry) -> {
String cryptsyMarketId = entry.getKey();
String marketName = entry.getValue();
Market market = dataService.getMarket(marketName);
if (notNull(market)) {
cryptsyExchange.converter.associateMarket(market, Integer.valueOf(cryptsyMarketId));
} else {
LOGGER.debug("associated market [+" + marketName + "] does not exist, removing from configuration");
markedForRemoval.add(cryptsyMarketId);
}
});
// Removing market associations missing from database
if (!markedForRemoval.isEmpty()) {
markedForRemoval.forEach((market) -> configuration.remove("exchangeMarketToMarket", market));
}
// Update configuration
configuration.save();
return cryptsyExchange;
}
// Lot of other code there
}
And of course the stackTrace:
Exception in thread "main" java.lang.ClassFormatError: Illegal local variable table length 288 in method com.jarvis.exchange.cryptsy.CryptsyExchange.build_aroundBody0(Ljava/lang/String;Lcom/jarvis/data/service/DataService;Lcom/jarvis/util/ConfigPathBuilder;Lorg/aspectj/lang/JoinPoint;)Lcom/jarvis/exchange/cryptsy/CryptsyExchange;
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2688)
at java.lang.Class.getDeclaredMethods(Class.java:1962)
at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:467)
at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:451)
at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:512)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:663)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1396)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:382)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:353)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:82)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:609)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
at com.jarvis.Jarvis.<clinit>(Jarvis.java:10)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:259)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:116)
I've tried this on any other class on my project (annotation can be applied on Type or method) and it worked, but exactly with this bean build method I'm facing issues and can't find any workaround. Maybe current support for Java 8 by Aspectj is buggy and it actually corrupts bytecode. Or perhaps there is something wrong I've done there?
Some questions:
Do you use full-blown AspectJ or just Spring AOP with #AspectJ syntax?
Do you compile with source/target level 1.8 or maybe still 1.7?
Does the exact same code work with Java 7 and AspectJ 1.8.0?
If not, does it work with Java 7 and AspectJ 1.7.4?
Can you please provide a stand-alone minimal code sample reproducing the problem? Maybe even on GitHub with a Maven build?) Many of your referenced classes are invisible to me, so I cannot reproducte it.
AJ 1.8.0 is brandnew and some of its problems are actually caused by ECJ (Eclipse Java compiler). Some have already been fixed, so you could try a current developer build (select "Last Known Good developer build".
Update:
I was able to reproduce the problem with a small code sample, independent of any other classes. The problem is not annotations or simple forEach lambdas, but obviously nested forEach lambdas.
I filed an AspectJ bug on http://bugs.eclipse.org/bugs/show_bug.cgi?id=435446.
Update 2:
The bug ticket also describes how to work around the problem by excluding lambda calls from the pointcut.
Another workaround I just found is to run the JVM with parameter -noverify.
Update 3:
The bugfix is done and available as a development build. It will be part of the upcoming AspectJ 1.8.1.
I've found that the problem comes from lambda expression usage :(
replacing all lambdas with regular for fixes the problem
exchangeCurrencyToCurrency.entrySet().stream().forEach((entry) -> {
String cryptsyCurrencyCode = entry.getKey();
String currencySymbol = entry.getValue();
com.jarvis.data.entity.Currency currency = dataService.getCurrency(currencySymbol);
if (notNull(currency)) {
associateCurrency(currency, cryptsyCurrencyCode);
} else {
LOGGER.debug("associated currency [" + currencySymbol + "] does not exist in database, removing from configuration");
markedForRemoval.add(cryptsyCurrencyCode);
}
});
result
for(Map.Entry<String, String> entry : exchangeCurrencyToCurrency.entrySet()){
String cryptsyCurrencyCode = entry.getKey();
String currencySymbol = entry.getValue();
com.jarvis.data.entity.Currency currency = dataService.getCurrency(currencySymbol);
if (notNull(currency)) {
associateCurrency(currency, cryptsyCurrencyCode);
} else {
LOGGER.debug("associated currency [" + currencySymbol + "] does not exist in database, removing from configuration");
markedForRemoval.add(cryptsyCurrencyCode);
}
}
Using eclipse's jdt refactoring framework, I am trying to convert two different code bases to the same names. They are almost identical codebases except that names are different.
Function/Field/Class renaming works fine, but when it comes to parameters it yells at me that the workbench is not created yet. However i'm trying to do this in a headless manor.
private void refactor(String task, IJavaElement element, String new_name) throws CoreException
{
RefactoringStatus status = new RefactoringStatus();
RefactoringContribution contrib = RefactoringCore.getRefactoringContribution(task);
RenameJavaElementDescriptor rnDesc = (RenameJavaElementDescriptor)contrib.createDescriptor();
rnDesc.setFlags(JavaRefactoringDescriptor.JAR_MIGRATION | JavaRefactoringDescriptor.JAR_REFACTORING);
rnDesc.setProject(element.getJavaProject().getProject().getName());
rnDesc.setUpdateReferences(true);
rnDesc.setJavaElement(element);
rnDesc.setNewName(new_name);
Refactoring ref = rnDesc.createRefactoring(status);
ref.checkInitialConditions(NULL_MON);
ref.checkFinalConditions(NULL_MON);
Change change = ref.createChange(NULL_MON);
change.perform(NULL_MON);
}
This works fine:
for (IMethod method : type.getMethods())
{
refactor(IJavaRefactorings.RENAME_METHOD, method, {new name});
}
This does not:
for (IMethod method : type.getMethods())
{
for (ILocalVariable param : method.getParameters())
{
refactor(IJavaRefactorings.RENAME_LOCAL_VARIABLE, param, {new name});
}
}
And the error, not really helpful as I said I need to do this in a headless manor {so can't make workbench}
java.lang.IllegalStateException: Workbench has not been created yet.
at org.eclipse.ui.PlatformUI.getWorkbench(PlatformUI.java:92)
at org.eclipse.jdt.internal.ui.javaeditor.ASTProvider.install(ASTProvider.java:245)
at org.eclipse.jdt.internal.ui.javaeditor.ASTProvider.<init>(ASTProvider.java:236)
at org.eclipse.jdt.internal.ui.JavaPlugin.getASTProvider(JavaPlugin.java:710)
at org.eclipse.jdt.ui.SharedASTProvider.getAST(SharedASTProvider.java:128)
at org.eclipse.jdt.internal.corext.refactoring.util.RefactoringASTParser.parseWithASTProvider(RefactoringASTParser.java:119)
at org.eclipse.jdt.internal.corext.refactoring.rename.RenameLocalVariableProcessor.initAST(RenameLocalVariableProcessor.java:231)
at org.eclipse.jdt.internal.corext.refactoring.rename.RenameLocalVariableProcessor.checkInitialConditions(RenameLocalVariableProcessor.java:218)
at org.eclipse.ltk.core.refactoring.participants.ProcessorBasedRefactoring.checkInitialConditions(ProcessorBasedRefactoring.java:203)
UPDATE: Made some progress, now I can refactor functions that are not overrides. But any function that overrides another or an interface screws up:
F_ARGUMENTS = JavaRefactoringDescriptor.class.getDeclaredField("fArguments");
F_ARGUMENTS.setAccessible(true);
private void refactor(IMethod method, String[] names) throws CoreException
{
/* My attempt to fix the interface issues, causes duplicate functions instead of renaming the parameters
IMethod parent = null;
if (method.getDeclaringType().isInterface())
{
parent = MethodChecks.overridesAnotherMethod(method, method.getDeclaringType().newSupertypeHierarchy(NULL_MON));
}
else if (MethodChecks.isVirtual(method))
{
ITypeHierarchy hierarchy = method.getDeclaringType().newTypeHierarchy(NULL_MON);
parent = MethodChecks.isDeclaredInInterface(method, hierarchy, NULL_MON);
if (parent == null)
{
parent = MethodChecks.overridesAnotherMethod(method, hierarchy);
}
}
parent = (parent == null ? method : parent);
if (!method.equals(parent))
{
refactor(parent, names);
return;
}*/
String task = IJavaRefactorings.CHANGE_METHOD_SIGNATURE;
RefactoringStatus status = new RefactoringStatus();
ChangeMethodSignatureRefactoringContribution contrib = (ChangeMethodSignatureRefactoringContribution)RefactoringCore.getRefactoringContribution(task);
ChangeMethodSignatureDescriptor desc = (ChangeMethodSignatureDescriptor)contrib.createDescriptor();
desc.setFlags(JavaRefactoringDescriptor.JAR_MIGRATION |
JavaRefactoringDescriptor.JAR_REFACTORING |
RefactoringDescriptor.MULTI_CHANGE |
RefactoringDescriptor.STRUCTURAL_CHANGE);
Map<String, String> args = null;
try
{
args = (Map<String, String>)F_ARGUMENTS.get(desc);
}
catch (Exception e)
{
e.printStackTrace();
}
String project = method.getJavaProject().getProject().getName();
desc.setProject(method.getJavaProject().getProject().getName());
args.put("input", JavaRefactoringDescriptorUtil.elementToHandle(project, method));
args.put("name", method.getElementName());
args.put("deprecate", "false");
args.put("delegate", "true");
boolean changed = false;
int x = 0;
for (ILocalVariable param : method.getParameters())
{
if (!param.getElementName().equals(names[x]))
{
changed = true;
}
String type = "String"; //Doesn't seem to actually matter as long as they are both the same
String info = type + " " + param.getElementName() + " " + x + " " +
type + " " + names[x] + " false";
args.put("parameter" + (x + 1), info);
x++;
}
if (changed)
{
refactor(desc.createRefactoring(status));
}
}
This is what I came up with:
ChangeSignatureProcessor changeSignatureProcessor = new ChangeSignatureProcessor((IMethod) node.resolveBinding().getJavaElement());
ParameterInfo info=new ParameterInfo("FormContext", "formContext", ParameterInfo.INDEX_FOR_ADDED);
info.setDefaultValue("formContext");
changeSignatureProcessor.getParameterInfos().add(0,info);
RefactoringStatus status = new RefactoringStatus();
CheckConditionsContext context= new CheckConditionsContext();
context.add(new ValidateEditChecker(null));
context.add(new ResourceChangeChecker());
changeSignatureProcessor.checkInitialConditions(monitor);
changeSignatureProcessor.checkFinalConditions(monitor,context);
changeSignatureProcessor.createChange(monitor).perform(monitor);
I am trying to write a test for a dropbox crawler.
I have setup a Fixture which loads data into the database on load.
Fixtures.deleteDatabase();
Fixtures.loadModels("testusers.yaml");
The problem occurs when the code
da = da.merge(); //exceptions gets thrown here
is reached. It throws an JdbcSQLException:
org.h2.jdbc.JdbcSQLException: Timeout trying to lock table "USER"; SQL statement:
select user0_.id as id7_0_, user0_.activationSent as activati2_7_0_,
user0_.confirmationCode as confirma3_7_0_, user0_.email as email7_0_,
user0_.first_name as first5_7_0_, user0_.isAdmin as isAdmin7_0_,
user0_.lastLogin as lastLogin7_0_, user0_.last_name as last8_7_0_,
user0_.passwordHash as password9_7_0_,
user0_.recoverPasswordCode as recover10_7_0_,
user0_.referralCode as referra11_7_0_,
user0_.signupDate as signupDate7_0_ from User user0_ where user0_.id=? [50200-149]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.get(DbException.java:144)
at org.h2.table.RegularTable.doLock(RegularTable.java:499)
at org.h2.table.RegularTable.lock(RegularTable.java:433)
at org.h2.table.TableFilter.lock(TableFilter.java:140)
at org.h2.command.dml.Select.queryWithoutCache(Select.java:571)
at org.h2.command.dml.Query.query(Query.java:257)
at org.h2.command.dml.Query.query(Query.java:227)
at org.h2.command.CommandContainer.query(CommandContainer.java:78)
at org.h2.command.Command.executeQuery(Command.java:178)
at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:96)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1953)
at org.hibernate.loader.Loader.doQuery(Loader.java:802)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
at org.hibernate.loader.Loader.loadEntity(Loader.java:2037)
... 36 more
UpdateDropboxRowJob.java where the exception gets thrown
package mashpan.crawl.jobs;
import play.Logger;
import play.jobs.*;
import models.DropboxAuthentication;
public class UpdateDropboxRowJob extends Job {
private DropboxAuthentication da;
private long count;
private boolean crawled;
public UpdateDropboxRowJob(DropboxAuthentication da, long count, boolean crawled) {
Logger.debug("[UpdateDropboxRowJob] ctor: "
+",[da=" + (da == null ? "null" : da.toString()) + "]"
+", count=[" + count + "]"
+", crawled=[" + crawled + "]"
);
this.da = da;
this.count = count;
this.crawled = crawled;
}
#Override
public void doJob() throws Exception {
Logger.debug("[UpdateDropboxRowJob] doJob: "
+"da=[" + (da == null ? "null" : da.toString()) + "]"
+"count=[" + count + "]"
+"crawled=[" + crawled + "]"
+"da.count"+(da == null ? "da==null" : da.count) + "]"
);
da = da.merge(); //exceptions gets thrown here
da.count = this.count;
da.crawled = this.crawled;
da.save();
}
}
#Entity
public class DropboxAuthentication extends Model {
public String type;
//public String email;
public String token_key;
public String token_secret;
public long count;
public boolean crawled;
public byte[] cipher;
public byte[] iv;
public long lastCrawlTime;
#OneToOne
public User user;
public DropboxAuthentication(String type, byte[] cipher, byte[] iv, String token_key, String token_secret, User u, Date issueDate)
{
this.type = type;
this.cipher = cipher;
this.iv = iv;
this.token_key = token_key;
this.token_secret = token_secret;
this.user = u;
}
#Override
public String toString() {
return "DropboxAuthentication [type=" + type + ", token_key="
+ token_key + ", token_secret=" + token_secret + ", count="
+ count + ", crawled=" + crawled + ", cipher="
+ Arrays.toString(cipher) + ", iv=" + Arrays.toString(iv)
+ ", lastCrawlTime=" + lastCrawlTime + ", user=" + user + "]";
}
}
I suppose that there is a problem with transactions, I tried to do
JPA.em().getTransaction().commit();
but I gives me another exception that the transaction is not activated. Suprisingly
JPA.em().getTransaction().begin();
JPA.em().getTransaction().commit();
throws an exception that there is already a transaction running.
So probably the main question is how play handles transactions in test mode
play framework: 1.2.3
*********** UPDATE *************
I changed my application.conf from
%db=mem
to
%test.db=mysql://USER:PASSWORD#localhost/development
In the following I get a different exception (at the same line as before) now:
javax.persistence.EntityNotFoundException: Unable to find models.User with id 1
at org.hibernate.ejb.Ejb3Configuration$Ejb3EntityNotFoundDelegate.handleEntityNotFound(Ejb3Configuration.java:133)
at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:233)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:285)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:152)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:1090)
at org.hibernate.impl.SessionImpl.internalLoad(SessionImpl.java:1038)
at org.hibernate.type.EntityType.resolveIdentifier(EntityType.java:630)
at org.hibernate.type.EntityType.resolve(EntityType.java:438)
at org.hibernate.type.EntityType.replace(EntityType.java:298)
at org.hibernate.type.AbstractType.replace(AbstractType.java:176)
at org.hibernate.type.TypeHelper.replace(TypeHelper.java:212)
at org.hibernate.event.def.DefaultMergeEventListener.copyValues(DefaultMergeEventListener.java:600)
at org.hibernate.event.def.DefaultMergeEventListener.mergeTransientEntity(DefaultMergeEventListener.java:337)
at org.hibernate.event.def.DefaultMergeEventListener.entityIsTransient(DefaultMergeEventListener.java:303)
at org.hibernate.event.def.DefaultMergeEventListener.onMerge(DefaultMergeEventListener.java:258)
at org.hibernate.event.def.DefaultMergeEventListener.onMerge(DefaultMergeEventListener.java:84)
at org.hibernate.impl.SessionImpl.fireMerge(SessionImpl.java:867)
at org.hibernate.impl.SessionImpl.merge(SessionImpl.java:851)
at org.hibernate.impl.SessionImpl.merge(SessionImpl.java:855)
at org.hibernate.ejb.AbstractEntityManagerImpl.merge(AbstractEntityManagerImpl.java:686)
at play.db.jpa.GenericModel.merge(GenericModel.java:211)
at mashpan.crawl.jobs.UpdateDropboxRowJob.doJob(UpdateDropboxRowJob.java:33)
at play.jobs.Job.doJobWithResult(Job.java:50)
at play.jobs.Job.call(Job.java:146)
However, looking in the db I see that the user is actually persisted there.
Is it possible that the job is called concurrently? or there is some other server code that is also writing to that DropboxAuthentication table?
If so, maybe you could try increasing the lock timeout for the H2 database by doing this to see if it makes a difference.
I'd like to be able to read in an XML schema (i.e. xsd) and from that know what are valid attributes, child elements, values as I walk through it.
For example, let's say I have an xsd that this xml will validate against:
<root>
<element-a type="something">
<element-b>blah</element-b>
<element-c>blahblah</element-c>
</element-a>
</root>
I've tinkered with several libraries and I can confidently get <root> as the root element. Beyond that I'm lost.
Given an element I need to know what child elements are required or allowed, attributes, facets, choices, etc. Using the above example I'd want to know that element-a has an attribute type and may have children element-b and element-c...or must have children element-b and element-c...or must have one of each...you get the picture I hope.
I've looked at numerous libraries such as XSOM, Eclipse XSD, Apache XmlSchema and found they're all short on good sample code. My search of the Internet has also been unsuccessful.
Does anyone know of a good example or even a book that demonstrates how to go through an XML schema and find out what would be valid options at a given point in a validated XML document?
clarification
I'm not looking to validate a document, rather I'd like to know the options at a given point to assist in creating or editing a document. If I know "I am here" in a document, I'd like to determing what I can do at that point. "Insert one of element A, B, or C" or "attach attribute 'description'".
This is a good question. Although, it is old, I did not find an acceptable answer. The thing is that the existing libraries I am aware of (XSOM, Apache XmlSchema) are designed as object models. The implementors did not have the intention to provide any utility methods — you should consider implement them yourself using the provided object model.
Let's see how querying context-specific elements can be done by the means of Apache XmlSchema.
You can use their tutorial as a starting point. In addition, Apache CFX framework provides the XmlSchemaUtils class with lots of handy code examples.
First of all, read the XmlSchemaCollection as illustrated by the library's tutorial:
XmlSchemaCollection xmlSchemaCollection = new XmlSchemaCollection();
xmlSchemaCollection.read(inputSource, new ValidationEventHandler());
Now, XML Schema defines two kinds of data types:
Simple types
Complex types
Simple types are represented by the XmlSchemaSimpleType class. Handling them is easy. Read the documentation: https://ws.apache.org/commons/XmlSchema/apidocs/org/apache/ws/commons/schema/XmlSchemaSimpleType.html. But let's see how to handle complex types. Let's start with a simple method:
#Override
public List<QName> getChildElementNames(QName parentElementName) {
XmlSchemaElement element = xmlSchemaCollection.getElementByQName(parentElementName);
XmlSchemaType type = element != null ? element.getSchemaType() : null;
List<QName> result = new LinkedList<>();
if (type instanceof XmlSchemaComplexType) {
addElementNames(result, (XmlSchemaComplexType) type);
}
return result;
}
XmlSchemaComplexType may stand for both real type and for the extension element. Please see the public static QName getBaseType(XmlSchemaComplexType type) method of the XmlSchemaUtils class.
private void addElementNames(List<QName> result, XmlSchemaComplexType type) {
XmlSchemaComplexType baseType = getBaseType(type);
XmlSchemaParticle particle = baseType != null ? baseType.getParticle() : type.getParticle();
addElementNames(result, particle);
}
When you handle XmlSchemaParticle, consider that it can have multiple implementations. See: https://ws.apache.org/commons/XmlSchema/apidocs/org/apache/ws/commons/schema/XmlSchemaParticle.html
private void addElementNames(List<QName> result, XmlSchemaParticle particle) {
if (particle instanceof XmlSchemaAny) {
} else if (particle instanceof XmlSchemaElement) {
} else if (particle instanceof XmlSchemaGroupBase) {
} else if (particle instanceof XmlSchemaGroupRef) {
}
}
The other thing to bear in mind is that elements can be either abstract or concrete. Again, the JavaDocs are the best guidance.
Many of the solutions for validating XML in java use the JAXB API. There's an extensive tutorial available here. The basic recipe for doing what you're looking for with JAXB is as follows:
Obtain or create the XML schema to validate against.
Generate Java classes to bind the XML to using xjc, the JAXB compiler.
Write java code to:
Open the XML content as an input stream.
Create a JAXBContext and Unmarshaller
Pass the input stream to the Unmarshaller's unmarshal method.
The parts of the tutorial you can read for this are:
Hello, world
Unmarshalling XML
I see you have tried Eclipse XSD. Have you tried Eclipse Modeling Framework (EMF)? You can:
Generating an EMF Model using XML Schema (XSD)
Create a dynamic instance from your metamodel (3.1 With the dynamic instance creation tool)
This is for exploring the xsd. You can create the dynamic instance of the root element then you can right click the element and create child element. There you will see what the possible children element and so on.
As for saving the created EMF model to an xml complied xsd: I have to look it up. I think you can use JAXB for that (How to use EMF to read XML file?).
Some refs:
EMF: Eclipse Modeling Framework, 2nd Edition (written by creators)
Eclipse Modeling Framework (EMF)
Discover the Eclipse Modeling Framework (EMF) and Its Dynamic Capabilities
Creating Dynamic EMF Models From XSDs and Loading its Instances From XML as SDOs
This is a fairly complete sample on how to parse an XSD using XSOM:
import java.io.File;
import java.util.Iterator;
import java.util.Vector;
import org.xml.sax.ErrorHandler;
import com.sun.xml.xsom.XSComplexType;
import com.sun.xml.xsom.XSElementDecl;
import com.sun.xml.xsom.XSFacet;
import com.sun.xml.xsom.XSModelGroup;
import com.sun.xml.xsom.XSModelGroupDecl;
import com.sun.xml.xsom.XSParticle;
import com.sun.xml.xsom.XSRestrictionSimpleType;
import com.sun.xml.xsom.XSSchema;
import com.sun.xml.xsom.XSSchemaSet;
import com.sun.xml.xsom.XSSimpleType;
import com.sun.xml.xsom.XSTerm;
import com.sun.xml.xsom.impl.Const;
import com.sun.xml.xsom.parser.XSOMParser;
import com.sun.xml.xsom.util.DomAnnotationParserFactory;
public class XSOMNavigator
{
public static class SimpleTypeRestriction
{
public String[] enumeration = null;
public String maxValue = null;
public String minValue = null;
public String length = null;
public String maxLength = null;
public String minLength = null;
public String[] pattern = null;
public String totalDigits = null;
public String fractionDigits = null;
public String whiteSpace = null;
public String toString()
{
String enumValues = "";
if (enumeration != null)
{
for(String val : enumeration)
{
enumValues += val + ", ";
}
enumValues = enumValues.substring(0, enumValues.lastIndexOf(','));
}
String patternValues = "";
if (pattern != null)
{
for(String val : pattern)
{
patternValues += "(" + val + ")|";
}
patternValues = patternValues.substring(0, patternValues.lastIndexOf('|'));
}
String retval = "";
retval += minValue == null ? "" : "[MinValue = " + minValue + "]\t";
retval += maxValue == null ? "" : "[MaxValue = " + maxValue + "]\t";
retval += minLength == null ? "" : "[MinLength = " + minLength + "]\t";
retval += maxLength == null ? "" : "[MaxLength = " + maxLength + "]\t";
retval += pattern == null ? "" : "[Pattern(s) = " + patternValues + "]\t";
retval += totalDigits == null ? "" : "[TotalDigits = " + totalDigits + "]\t";
retval += fractionDigits == null ? "" : "[FractionDigits = " + fractionDigits + "]\t";
retval += whiteSpace == null ? "" : "[WhiteSpace = " + whiteSpace + "]\t";
retval += length == null ? "" : "[Length = " + length + "]\t";
retval += enumeration == null ? "" : "[Enumeration Values = " + enumValues + "]\t";
return retval;
}
}
private static void initRestrictions(XSSimpleType xsSimpleType, SimpleTypeRestriction simpleTypeRestriction)
{
XSRestrictionSimpleType restriction = xsSimpleType.asRestriction();
if (restriction != null)
{
Vector<String> enumeration = new Vector<String>();
Vector<String> pattern = new Vector<String>();
for (XSFacet facet : restriction.getDeclaredFacets())
{
if (facet.getName().equals(XSFacet.FACET_ENUMERATION))
{
enumeration.add(facet.getValue().value);
}
if (facet.getName().equals(XSFacet.FACET_MAXINCLUSIVE))
{
simpleTypeRestriction.maxValue = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_MININCLUSIVE))
{
simpleTypeRestriction.minValue = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_MAXEXCLUSIVE))
{
simpleTypeRestriction.maxValue = String.valueOf(Integer.parseInt(facet.getValue().value) - 1);
}
if (facet.getName().equals(XSFacet.FACET_MINEXCLUSIVE))
{
simpleTypeRestriction.minValue = String.valueOf(Integer.parseInt(facet.getValue().value) + 1);
}
if (facet.getName().equals(XSFacet.FACET_LENGTH))
{
simpleTypeRestriction.length = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_MAXLENGTH))
{
simpleTypeRestriction.maxLength = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_MINLENGTH))
{
simpleTypeRestriction.minLength = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_PATTERN))
{
pattern.add(facet.getValue().value);
}
if (facet.getName().equals(XSFacet.FACET_TOTALDIGITS))
{
simpleTypeRestriction.totalDigits = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_FRACTIONDIGITS))
{
simpleTypeRestriction.fractionDigits = facet.getValue().value;
}
if (facet.getName().equals(XSFacet.FACET_WHITESPACE))
{
simpleTypeRestriction.whiteSpace = facet.getValue().value;
}
}
if (enumeration.size() > 0)
{
simpleTypeRestriction.enumeration = enumeration.toArray(new String[] {});
}
if (pattern.size() > 0)
{
simpleTypeRestriction.pattern = pattern.toArray(new String[] {});
}
}
}
private static void printParticle(XSParticle particle, String occurs, String absPath, String indent)
{
boolean repeats = particle.isRepeated();
occurs = " MinOccurs = " + particle.getMinOccurs() + ", MaxOccurs = " + particle.getMaxOccurs() + ", Repeats = " + Boolean.toString(repeats);
XSTerm term = particle.getTerm();
if (term.isModelGroup())
{
printGroup(term.asModelGroup(), occurs, absPath, indent);
}
else if(term.isModelGroupDecl())
{
printGroupDecl(term.asModelGroupDecl(), occurs, absPath, indent);
}
else if (term.isElementDecl())
{
printElement(term.asElementDecl(), occurs, absPath, indent);
}
}
private static void printGroup(XSModelGroup modelGroup, String occurs, String absPath, String indent)
{
System.out.println(indent + "[Start of Group " + modelGroup.getCompositor() + occurs + "]" );
for (XSParticle particle : modelGroup.getChildren())
{
printParticle(particle, occurs, absPath, indent + "\t");
}
System.out.println(indent + "[End of Group " + modelGroup.getCompositor() + "]");
}
private static void printGroupDecl(XSModelGroupDecl modelGroupDecl, String occurs, String absPath, String indent)
{
System.out.println(indent + "[GroupDecl " + modelGroupDecl.getName() + occurs + "]");
printGroup(modelGroupDecl.getModelGroup(), occurs, absPath, indent);
}
private static void printComplexType(XSComplexType complexType, String occurs, String absPath, String indent)
{
System.out.println();
XSParticle particle = complexType.getContentType().asParticle();
if (particle != null)
{
printParticle(particle, occurs, absPath, indent);
}
}
private static void printSimpleType(XSSimpleType simpleType, String occurs, String absPath, String indent)
{
SimpleTypeRestriction restriction = new SimpleTypeRestriction();
initRestrictions(simpleType, restriction);
System.out.println(restriction.toString());
}
public static void printElement(XSElementDecl element, String occurs, String absPath, String indent)
{
absPath += "/" + element.getName();
String typeName = element.getType().getBaseType().getName();
if(element.getType().isSimpleType() && element.getType().asSimpleType().isPrimitive())
{
// We have a primitive type - So use that instead
typeName = element.getType().asSimpleType().getPrimitiveType().getName();
}
boolean nillable = element.isNillable();
System.out.print(indent + "[Element " + absPath + " " + occurs + "] of type [" + typeName + "]" + (nillable ? " [nillable] " : ""));
if (element.getType().isComplexType())
{
printComplexType(element.getType().asComplexType(), occurs, absPath, indent);
}
else
{
printSimpleType(element.getType().asSimpleType(), occurs, absPath, indent);
}
}
public static void printNameSpace(XSSchema s, String indent)
{
String nameSpace = s.getTargetNamespace();
// We do not want the default XSD namespaces or a namespace with nothing in it
if(nameSpace == null || Const.schemaNamespace.equals(nameSpace) || s.getElementDecls().isEmpty())
{
return;
}
System.out.println("Target namespace: " + nameSpace);
Iterator<XSElementDecl> jtr = s.iterateElementDecls();
while (jtr.hasNext())
{
XSElementDecl e = (XSElementDecl) jtr.next();
String occurs = "";
String absPath = "";
XSOMNavigator.printElement(e, occurs, absPath,indent);
System.out.println();
}
}
public static void xsomNavigate(File xsdFile)
{
ErrorHandler errorHandler = new ErrorReporter(System.err);
XSSchemaSet schemaSet = null;
XSOMParser parser = new XSOMParser();
try
{
parser.setErrorHandler(errorHandler);
parser.setAnnotationParser(new DomAnnotationParserFactory());
parser.parse(xsdFile);
schemaSet = parser.getResult();
}
catch (Exception exp)
{
exp.printStackTrace(System.out);
}
if(schemaSet != null)
{
// iterate each XSSchema object. XSSchema is a per-namespace schema.
Iterator<XSSchema> itr = schemaSet.iterateSchema();
while (itr.hasNext())
{
XSSchema s = (XSSchema) itr.next();
String indent = "";
printNameSpace(s, indent);
}
}
}
public static void printFile(String fileName)
{
File fileToParse = new File(fileName);
if (fileToParse != null && fileToParse.canRead())
{
xsomNavigate(fileToParse);
}
}
}
And for your Error Reporter use:
import java.io.OutputStream;
import java.io.PrintStream;
import java.text.MessageFormat;
import org.xml.sax.ErrorHandler;
import org.xml.sax.SAXException;
import org.xml.sax.SAXParseException;
public class ErrorReporter implements ErrorHandler {
private final PrintStream out;
public ErrorReporter( PrintStream o ) { this.out = o; }
public ErrorReporter( OutputStream o ) { this(new PrintStream(o)); }
public void warning(SAXParseException e) throws SAXException {
print("[Warning]",e);
}
public void error(SAXParseException e) throws SAXException {
print("[Error ]",e);
}
public void fatalError(SAXParseException e) throws SAXException {
print("[Fatal ]",e);
}
private void print( String header, SAXParseException e ) {
out.println(header+' '+e.getMessage());
out.println(MessageFormat.format(" line {0} at {1}",
new Object[]{
Integer.toString(e.getLineNumber()),
e.getSystemId()}));
}
}
For your main use:
public class WDXSOMParser
{
public static void main(String[] args)
{
String fileName = null;
if(args != null && args.length > 0 && args[0] != null)
fileName = args[0];
else
fileName = "C:\\xml\\CollectionComments\\CollectionComment1.07.xsd";
//fileName = "C:\\xml\\PropertyListingContractSaleInfo\\PropertyListingContractSaleInfo.xsd";
//fileName = "C:\\xml\\PropertyPreservation\\PropertyPreservation.xsd";
XSOMNavigator.printFile(fileName);
}
}
It's agood bit of work depending on how compex your xsd is but basically.
if you had
<Document>
<Header/>
<Body/>
<Document>
And you wanted to find out where were the alowable children of header you'd (taking account of namespaces)
Xpath would have you look for '/element[name="Document"]/element[name="Header"]'
After that it depends on how much you want to do. You might find it easier to write or find something that loads an xsd into a DOM type structure.
Course you are going to possibly find all sorts of things under that elment in xsd, choice, sequence, any, attributes, complexType, SimpleContent, annotation.
Loads of time consuming fun.
Have a look at this.
How to parse schema using XOM Parser.
Also, here is the project home for XOM