Develop Reference

How to convert ArrayList to ExampleSet in Rapidminer?

I'm creating an extension for rapidminer using java. I have an array of elements of type Example and I need to covert it to a dataset of type ExampleSet.
Rapidminer's ExampleSet definition looks like this:
public interface ExampleSet extends ResultObject, Cloneable, Iterable<Example>
I need to pick certain elements from dataset and send it back, still as ExampleSet, however casting is not working and I can't simply create new ExampleSet object since it's an interface.
private ExampleSet generateSet(ExampleSet dataset){
List<Example> list = new ArrayList<Example>();
// pick elements from sent dataset and add them to newly created list above
return (ExampleSet)list;
}

You will need more than a simple explicit cast.
In RapidMiner, an ExampleSet is not just a collection of Example. It contains more complex information and logic.
Therefore, you need another approach to work with ExampleSets. Like you already said, it is just the interface, which lead us to choice of the right subtype.
For starters, (Since: 7.3) simply use one of ExampleSets class's methods .
You also need to define each Attribute this ExampleSet is going to have, namely the columns.
Below, I create one with a single Attribute called First
Attribute attributeFirst = AttributeFactory.createAttribute("First", Ontology.POLYNOMINAL);
ExampleSetBuilder builder = ExampleSets.from(attributeFirst);
builder.addDataRow(example.getDataRow());
ExampleSet result = builder.build();
You can also get the Attributes in a more generic way using:
Attribute[] attributes = example.getAttributes().createRegularAttributeArray();
ExampleSetBuilder builder = ExampleSets.from(attributes);
...
If you have many cases where you have to create or alter ExampleSet, I encourage you to write your own ExampleSetBuilder since the original implementation have many drawbacks.
You can also try searching for other extensions, which may already meet your requirements, and you do not need to create one of your own (belive me, it's not Headache-free).

the ExampleSet class is getting deprecated (but still perfectly fine to use).
You might want to consider switching over to the newer data set API called Belt (https://github.com/rapidminer/belt). It's faster and more intuitive to use. It's still actively developed, so feedback is also welcome.
Also if you have more specific questions, feel free to drop by the RapidMiner community (https://community.rapidminer.com/), where also many of the developers are very active.

Uni/Bi direction X One/Many X Many/One Association relationships

Following question also refers to discussion in following questions as well
https://stackoverflow.com/search?page=2&tab=Relevance&q=one%20to%20many%20unidirectional%20java
Best practise for adding a bidirectional relation in OO model
I tried to implementing 8 association combinations formed by [Unidirectional/Bidirectional] X [(One/Many) to (One/Many)] in Java. I found two cases can not be implemented namely Unidirectional One to One and Unidirectional One to Many (e.g. Person->*Vehicle). Other 6 combinations and Composition are possible programatically.
I Feel its not only the case with Java, these 2 cases do not exist. e.g. Use case - allocate one Aadhar/SSN number to only one person is possible if we know that number is not allocated to anybody else (reverse navigation is must). Does this mean we need to take care while making our design model not to arrive at these specific associations (though they might be present in analysis model)? I am confused on this.

Basic (No Aggregation)
If you are looking at basic unidirectional association, then that's the simplest of them all.
Unidirectional One to One
class Person {
String name;
}
Unidirectional One to Many
class Person {
List vehicles;
}
Composite Aggregation
If I assume that you are asking about composite relationshions (where one SSN can be assigned to at most one person), then you can still implement it.
How exactly you decide to implement it is however subject to your specific domain or e.g. how you store your data, because
reverse navigation is must
is not actually true, because you can just check all Person instances; or you can store all the SSNs in a smart data structure that allows you to quickly check if a new one is unique, and then you would assign it to the Person without additional checks, because you already know that it is unique).
Or you can implement also the opposite lookup, which is not prohibited even if the association is "uni-directional"
To quote the UML Specs (11.5.3.1 Associations) [emphasis mine]:
Navigability means that instances participating in links at runtime (instances of an Association) can be
accessed efficiently from instances at the other ends of the Association. The precise mechanism by which such efficient
access is achieved is implementation specific. If an end is not navigable, access from the other ends may or may not be
possible, and if it is, it might not be efficient.
Update from comments
Noone claims that upholding the relationship constraints has to be done in the accessors. In fact pretty much always you will have temporarily invalid relationships, imagine:
person = new Person();
// right now person is invalid state because it doesn't have an SSN
ssn = ssnGenerator.createNew();
// now ssn is also in invalid state because it has no person
person.setSSN(ssn);
// only now is person and ssn valid
(creating a constructor wouldn't help, because constructor is called after the object has already been created (so another part of the constructor could need the ssn already set).
So it is the responsibility of the programmer to ensure that the system upholds all constraints in whatever way it makes most sense. Using constructors/accessors is the easiest way in some circumstances, but you could e.g. wrap the code above in an atomic transaction. After all, if you kept your validation in the setSSN(), then what would happen if the programmer were to forget to call the method at all?
(person 1->* vehicle)
p1.add(v1) and p2.add(v1) are possible violations
You asked about "person ->* vehicle", now you've changed it to "person 1 -> * vehicle" so obviously the answer differs. But the same principle as above applies -- it is the responsibility of the system to uphold all constraints, and wherever that's done in accessors, validation methods, or the way the system constructed is an implementational detail -- there's no single best way, and there will be always trade-offs.

How to get over limitations of the Hibernate Criteria and Example APIs?

I'm in a position where our company has a database search service that is highly configurable, for which it's very useful to configure queries in a programmatic fashion. The Criteria API is powerful but when one of our developers refactors one of the data objects, the criteria restrictions won't signal that they're broken until we run our unit tests, or worse, are live and on our production environment. Recently, we had a refactoring project essentially double in working time unexpectedly due to this problem, a gap in project planning that, had we known how long it would really take, we probably would have taken an alternative approach.
I'd like to use the Example API to solve this problem. The Java compiler can loudly indicate that our queries are borked if we are specifying 'where' conditions on real POJO properties. However, there's only so much functionality in the Example API and it's limiting in many ways. Take the following example
Product product = new Product();
product.setName("P%");
Example prdExample = Example.create(product);
prdExample.excludeProperty("price");
prdExample.enableLike();
prdExample.ignoreCase();
Here, the property "name" is being queried against (where name like 'P%'), and if I were to remove or rename the field "name", we would know instantly. But what about the property "price"? It's being excluded because the Product object has some default value for it, so we're passing the "price" property name to an exclusion filter. Now if "price" got removed, this query would be syntactically invalid and you wouldn't know until runtime. LAME.
Another problem - what if we added a second where clause:
product.setPromo("Discounts up to 10%");
Because of the call to enableLike(), this example will match on the promo text "Discounts up to 10%", but also "Discounts up to 10,000,000 dollars" or anything else that matches. In general, the Example object's query-wide modifications, such as enableLike() or ignoreCase() aren't always going to be applicable to every property being checked against.
Here's a third, and major, issue - what about other special criteria? There's no way to get every product with a price greater than $10 using the standard example framework. There's no way to order results by promo, descending. If the Product object joined on some Manufacturer, there's no way to add a criterion on the related Manufacturer object either. There's no way to safely specify the FetchMode on the criteria for the Manufacturer either (although this is a problem with the Criteria API in general - invalid fetched relationships fail silently, even more of a time bomb)
For all of the above examples, you would need to go back to the Criteria API and use string representations of properties to make the query - again, eliminating the biggest benefit of Example queries.
What alternatives exist to the Example API that can get the kind of compile-time advice we need?

My company gives developers days when we can experiment and work on pet projects (a la Google) and I spent some time working on a framework to use Example queries while geting around the limitations described above. I've come up with something that could be useful to other people interested in Example queries too. Here is a sample of the framework using the Product example.
Criteria criteriaQuery = session.createCriteria(Product.class);
Restrictions<Product> restrictions = Restrictions.create(Product.class);
Product example = restrictions.getQueryObject();
example.setName(restrictions.like("N%"));
example.setPromo("Discounts up to 10%");
restrictions.addRestrictions(criteriaQuery);
Here's an attempt to fix the issues in the code example from the question - the problem of the default value for the "price" field no longer exists, because this framework requires that criteria be explicitly set. The second problem of having a query-wide enableLike() is gone - the matcher is only on the "name" field.
The other problems mentioned in the question are also gone in this framework. Here are example implementations.
product.setPrice(restrictions.gt(10)); // price > 10
product.setPromo(restrictions.order(false)); // order by promo desc
Restrictions<Manufacturer> manufacturerRestrictions
= Restrictions.create(Manufacturer.class);
//configure manuf restrictions in the same manner...
product.setManufacturer(restrictions.join(manufacturerRestrictions));
/* there are also joinSet() and joinList() methods
for one-to-many relationships as well */
Even more sophisticated restrictions are available.
product.setPrice(restrictions.between(45,55));
product.setManufacturer(restrictions.fetch(FetchMode.JOIN));
product.setName(restrictions.or("Foo", "Bar"));
After showing the framework to a coworker, he mentioned that many data mapped objects have private setters, making this kind of criteria setting difficult as well (a different problem with the Example API!). So, I've accounted for that too. Instead of using setters, getters are also queryable.
restrictions.is(product.getName()).eq("Foo");
restrictions.is(product.getPrice()).gt(10);
restrictions.is(product.getPromo()).order(false);
I've also added some extra checking on the objects to ensure better type safety - for example, the relative criteria (gt, ge, le, lt) all require a value ? extends Comparable for the parameter. Also, if you use a getter in the style specified above, and there's a #Transient annotation present on the getter, it will throw a runtime error.
But wait, there's more!
If you like that Hibernate's built-in Restrictions utility can be statically imported, so that you can do things like criteria.addRestriction(eq("name", "foo")) without making your code really verbose, there's an option for that too.
Restrictions<Product> restrictions = new Restrictions<Product>(){
public void query(Product queryObject){
queryObject.setPrice(gt(10));
queryObject.setPromo(order(false));
//gt() and order() inherited from Restrictions
}
}
That's it for now - thank you very much in advance for any feedback! We've posted the code on Sourceforge for those that are interested. http://sourceforge.net/projects/hqbe2/

The API looks great!
Restrictions.order(boolean) smells like control coupling. It's a little unclear what the values of the boolean argument represent.
I suggest replacing or supplementing with orderAscending() and orderDescending().

Have a look at Querydsl. Their JPA/Hibernate module requires code generation. Their Java collections module uses proxies but cannot be used with JPA/Hibernate at the moment.

Java classes with dynamic fields

I'm looking for clever ways to build dynamic Java classes, that is classes where you can add/remove fields at runtime. Usage scenario: I have an editor where users should be able to add fields to the model at runtime or maybe even create the whole model at runtime.
Some design goals:
Type safe without casts if possible for custom code that works on the dynamic fields (that code would come from plugins which extend the model in unforeseen ways).
Good performance (can you beat HashMap? Maybe use an array and assign indexes to the fields during setup?)
Field "reuse" (i.e. if you use the same type of field in several places, it should be possible to define it once and then reuse it).
Calculated fields which depend on the value of other fields
Signals should be sent when fields change value (no necessarily via the Beans API)
"Automatic" parent child relations (when you add a child to a parent, then the parent pointer in the child should be set for "free").
Easy to understand
Easy to use
Note that this is a "think outside the circle" question. I'll post an example below to get you in the mood :-)

Type safe without casts if possible for custom code that works on the dynamic fields (that code would come from plugins which extend the model in unforeseen ways)
AFAIK, this is not possible. You can only get type-safety without type casts if you use static typing. Static typing means method signatures (in classes or interfaces) that are known at compile time.
The best you can do is have an interface with a bunch of methods like String getStringValue(String field), int getIntValue(String field) and so on. And of course you can only do that for a predetermined set of types. Any field whose type is not in that set will require a typecast.

The obvious answer is to use a HashMap (or a LinkedHashMap if you care for the order of fields). Then, you can add dynamic fields via a get(String name) and a set(String name, Object value) method.
This code can be implemented in a common base class. Since there are only a few methods, it's also simple to use delegation if you need to extend something else.
To avoid the casting issue, you can use a type-safe object map:
TypedMap map = new TypedMap();
String expected = "Hallo";
map.set( KEY1, expected );
String value = map.get( KEY1 ); // Look Ma, no cast!
assertEquals( expected, value );
List<String> list = new ArrayList<String> ();
map.set( KEY2, list );
List<String> valueList = map.get( KEY2 ); // Even with generics
assertEquals( list, valueList );
The trick here is the key which contains the type information:
TypedMapKey<String> KEY1 = new TypedMapKey<String>( "key1" );
TypedMapKey<List<String>> KEY2 = new TypedMapKey<List<String>>( "key2" );
The performance will be OK.
Field reuse is by using the same value type or by extending the key class of the type-safe object map with additional functionality.
Calculated fields could be implemented with a second map that stores Future instances which do the calculation.
Since all the manipulation happens in just two (or at least a few) methods, sending signals is simple and can be done any way you like.
To implement automatic parent/child handling, install a signal listener on the "set parent" signal of the child and then add the child to the new parent (and remove it from the old one if necessary).
Since no framework is used and no tricks are necessary, the resulting code should be pretty clean and easy to understand. Not using String as keys has the additional benefit that people won't litter the code with string literals.

So basically you're trying to create a new kind of object model with more dynamic properties, a bit like a dynamic language?
Might be worth looking at the source code for Rhino (i.e. Javascript implemented in Java), which faces a similar challenge of implementing a dynamic type system in Java.
Off the top of my head, I suspect you will find that internal HashMaps ultimately work best for your purposes.
I wrote a little game (Tyrant - GPL source available) using a similar sort of dynamic object model featuring HashMaps, it worked great and performance was not an issue. I used a few tricks in the get and set methods to allow dynamic property modifiers, I'm sure you could do the same kind of thing to implement your signals and parent/child relations etc.
[EDIT] See the source of BaseObject how it is implemented.

You can use the bytecode manipulation libraries for it. Shortcoming of this approach is that you need to do create own classloader to load changes in classes dynamically.

I do almost the same, it's pure Java solution:
Users generate their own models, which are stored as JAXB schema.
Schema is compiled in Java classes on the fly and stored in
user jars
All classes are forced to extend one "root" class, where you could put every extra functionality you want.
Appropriate classloaders are implemented with "model change"
listeners.
Speaking of performance (which is important in my case), you can hardly beat this solution. Reusability is the same of XML document.

How do you query object collections in Java (Criteria/SQL-like)?

Suppose you have a collection of a few hundred in-memory objects and you need to query this List to return objects matching some SQL or Criteria like query. For example, you might have a List of Car objects and you want to return all cars made during the 1960s, with a license plate that starts with AZ, ordered by the name of the car model.
I know about JoSQL, has anyone used this, or have any experience with other/homegrown solutions?

Filtering is one way to do this, as discussed in other answers.
Filtering is not scalable though. On the surface time complexity would appear to be O(n) (i.e. already not scalable if the number of objects in the collection will grow), but actually because one or more tests need to be applied to each object depending on the query, time complexity more accurately is O(n t) where t is the number of tests to apply to each object.
So performance will degrade as additional objects are added to the collection, and/or as the number of tests in the query increases.
There is another way to do this, using indexing and set theory.
One approach is to build indexes on the fields within the objects stored in your collection and which you will subsequently test in your query.
Say you have a collection of Car objects and every Car object has a field color. Say your query is the equivalent of "SELECT * FROM cars WHERE Car.color = 'blue'". You could build an index on Car.color, which would basically look like this:
'blue' -> {Car{name=blue_car_1, color='blue'}, Car{name=blue_car_2, color='blue'}}
'red' -> {Car{name=red_car_1, color='red'}, Car{name=red_car_2, color='red'}}
Then given a query WHERE Car.color = 'blue', the set of blue cars could be retrieved in O(1) time complexity. If there were additional tests in your query, you could then test each car in that candidate set to check if it matched the remaining tests in your query. Since the candidate set is likely to be significantly smaller than the entire collection, time complexity is less than O(n) (in the engineering sense, see comments below). Performance does not degrade as much, when additional objects are added to the collection. But this is still not perfect, read on.
Another approach, is what I would refer to as a standing query index. To explain: with conventional iteration and filtering, the collection is iterated and every object is tested to see if it matches the query. So filtering is like running a query over a collection. A standing query index would be the other way around, where the collection is instead run over the query, but only once for each object in the collection, even though the collection could be queried any number of times.
A standing query index would be like registering a query with some sort of intelligent collection, such that as objects are added to and removed from the collection, the collection would automatically test each object against all of the standing queries which have been registered with it. If an object matches a standing query then the collection could add/remove it to/from a set dedicated to storing objects matching that query. Subsequently, objects matching any of the registered queries could be retrieved in O(1) time complexity.
The information above is taken from CQEngine (Collection Query Engine). This basically is a NoSQL query engine for retrieving objects from Java collections using SQL-like queries, without the overhead of iterating through the collection. It is built around the ideas above, plus some more. Disclaimer: I am the author. It's open source and in maven central. If you find it helpful please upvote this answer!

I have used Apache Commons JXPath in a production application. It allows you to apply XPath expressions to graphs of objects in Java.

yes, I know it's an old post, but technologies appear everyday and the answer will change in the time.
I think this is a good problem to solve it with LambdaJ. You can find it here:
http://code.google.com/p/lambdaj/
Here you have an example:
LOOK FOR ACTIVE CUSTOMERS // (Iterable version)
List<Customer> activeCustomers = new ArrayList<Customer>();
for (Customer customer : customers) {
if (customer.isActive()) {
activeCusomers.add(customer);
}
}
LambdaJ version
List<Customer> activeCustomers = select(customers,
having(on(Customer.class).isActive()));
Of course, having this kind of beauty impacts in the performance (a little... an average of 2 times), but can you find a more readable code?
It has many many features, another example could be sorting:
Sort Iterative
List<Person> sortedByAgePersons = new ArrayList<Person>(persons);
Collections.sort(sortedByAgePersons, new Comparator<Person>() {
public int compare(Person p1, Person p2) {
return Integer.valueOf(p1.getAge()).compareTo(p2.getAge());
}
});
Sort with lambda
List<Person> sortedByAgePersons = sort(persons, on(Person.class).getAge());
Update: after java 8 you can use out of the box lambda expressions, like:
List<Customer> activeCustomers = customers.stream()
.filter(Customer::isActive)
.collect(Collectors.toList());

Continuing the Comparator theme, you may also want to take a look at the Google Collections API. In particular, they have an interface called Predicate, which serves a similar role to Comparator, in that it is a simple interface that can be used by a filtering method, like Sets.filter. They include a whole bunch of composite predicate implementations, to do ANDs, ORs, etc.
Depending on the size of your data set, it may make more sense to use this approach than a SQL or external relational database approach.

If you need a single concrete match, you can have the class implement Comparator, then create a standalone object with all the hashed fields included and use it to return the index of the match. When you want to find more than one (potentially) object in the collection, you'll have to turn to a library like JoSQL (which has worked well in the trivial cases I've used it for).
In general, I tend to embed Derby into even my small applications, use Hibernate annotations to define my model classes and let Hibernate deal with caching schemes to keep everything fast.

I would use a Comparator that takes a range of years and license plate pattern as input parameters. Then just iterate through your collection and copy the objects that match. You'd likely end up making a whole package of custom Comparators with this approach.

The Comparator option is not bad, especially if you use anonymous classes (so as not to create redundant classes in the project), but eventually when you look at the flow of comparisons, it's pretty much just like looping over the entire collection yourself, specifying exactly the conditions for matching items:
if (Car car : cars) {
if (1959 < car.getYear() && 1970 > car.getYear() &&
car.getLicense().startsWith("AZ")) {
result.add(car);
}
}
Then there's the sorting... that might be a pain in the backside, but luckily there's class Collections and its sort methods, one of which receives a Comparator...