Develop Reference

Bulk operation for creating edges in gremlin

I have a fairly demanding task and unfortunately I can't get any further. Maybe you have a tip for me:
Goal:
Create a lot of edges with apache gremlin with only with one message
to the gremlin server (kind of bulk operation for creating edges).
The sourceId, the targetId and the type is saved in a list of JAVA pojos.
Use gremlin for java
Do not use IDs from the underlying engine, use some constant property PROP_ID for storing the user-given id
My current approach was:
create a list of map because gremlin java can only inject objects when they're maps or arrays
Object[] edgesMap = edges.stream().map(edge -> {
Map<String, String> m = new HashMap<>();
m.put("sourceId", edge.sourceId);
m.put("targetId", edge.targetId);
m.put("type", edge.type);
return m;
}).collect(Collectors.toList()).toArray();
Now i wanted to inject the object into a traversal and iterate the map. For every value i wanted to create an edge.
GraphTraversal<Vertex, Vertex> traversal = g.withSideEffect("edgeList", edgesMap).V().limit(1).sideEffect(
select("edgeList").unfold().as("edge").sideEffect(
g.V().has(PROP_ID, select("edge").select("targetId")).addE(select("edge").select("type")).from(select("edge").select("sourceId"))
)
);
traversal.iterate();
But unfortunately i cannot use .has in the anonymous traversal because .select(...).select(...) does not inject some constant value but returns a traversal. So i was told in the tinkerpop community that the has-traversal will always be true for every node and as a result for every node some edge is created.
I was told to use the where()-traversal to only get the node that fits with the property PROP_ID and the iterated value from the map. But the where()-function expects some P<String> predicate and with my current knowledge, i'm unable to get the select(...)-values into that predicate.
Maybe someone can help me so that I can either rewrite the traversal or someone has an idea how I can implement the requirements. Thanks! :)

What's an intelligent way of creating a graph like object for the Ford-Fulkerson algorithm?

I'm trying to implement a Ford–Fulkerson algorithm in java and I've been having some problems where my code gets obnoxiously and unnecessarily complicated.
What I do want is to have:
class Node:
private int id
private static int idAssigner // I may move this to another class
// etc
class flowNetwork
private Node source // begin point
private Node sink // end point
Now I want to group nodes similarly how I would a (bidirectional) tree. Each node has a list of all nodes it's connected to.
My problem is this: How could I give this connection a value (maximum flow, current flow) ?
Should I make another class Connection that has Node A Node B and max flow / current flow. And if I do that, how should I connect the nodes ? (as in should every node have a Connection and wouldn't that be redundant ? I'm a bit stuck.
edit Or should I just have Connections and implement some sort of search function to acomodate linking elements. It's all I can think of to be honest.
P.S.
This class is mostly just the math part, so I have never implemented a graph, nor does the course cover this, so thank you for helping a novice :) (that's if this doesn't get closed in like 5 minutes).

I think, you can use map of linked nodes in each node. With node key and link information as value.
It's not a fast solution, but it's simple.
Faster will be to have a matrix, elements of wich is a link objects, containing all link info. Rows and columns will be node indices.

Fast way to count the number of relationships for a node

I am trying to count the number of outgoing relationships of a particular type a node has. My code currently looks like this:
int count = 0;
for (Relationship r : node.getRelationships(RelationshipTypes.MODIFIES, Direction.OUTGOING))
{
count++;
}
return count;
The return type of getRelationships is Iterable so I can't use size() or equivalent. I am trying to avoid having to pull every relationship out of the database because some nodes have lots of relationships ( > 5 million). Is there a faster way of doing this?

No. The way neo4j stores relationships on disk for a node is in a linked list, and they do not keep any type of statistics for nodes or relationships. In order to get a count, you will have to go through all relationships for the node, of that type.
Even if you have a cache, with which they store it more efficiently, the system may still not provide a full picture. You method is the best method.

I would try to store outgoing in a data structure and get the size of the structure. This may take more time when the objects are initialized but it seems like the easiest way to quickly get size.

if node.getRelationships(RelationshipTypes.MODIFIES, Direction.OUTGOING) is returning a type of Collection then
to know the number of outgoing relationships of a particular type a node has , you can simply use the following :
int count = node.getRelationships(RelationshipTypes.MODIFIES, Direction.OUTGOING).size();

I see you are using the neo4j api. The other way would be to go with ThinkerPop gremlin query language which is available both for groovy and scala but they will do the same thing internally. As i know neo4j is giving you access trough an iterator because of performance reasons. For instance you could have million relationships but you want to paginate trough the results on the fly. It would be really be slow if Neo4J would return always a collection of relationships. That's why he returns a iterator and gives you access on the fly to the relationships. They are not retrieved from the DB until you need them.
So i would say NO. I hope i could help you.

Utility for graph manipulation in java

I needed a graph structure of key ==>> value such as following image:
Numbers in circle are key of its node.
I wanted access to stored value in key 2-7-6-5 and I wanted by 2-7 key retrieve a sub-graph contains collectin of 2, 6-5, 6-11 keys-values , so I wrote my implementation by nested maps and it worked fine but my question is :
Is there any custom Map implementation or third-party library for solve my situation for cleanup my code from manipulation manually such as String.split or loop and condition statements?

If you are really just looking for a 3rd-Party Java Library to work with graphs take a look at JUNG it has plenty of features for graph manipulation. However, it might be overkill for what you are trying to achieve.

take this one - really good for graph manipulations, and also for dispaying graph structure in swing
<dependency>
<groupId>jgraph</groupId>
<artifactId>jgraph</artifactId>
<version>5.13.0.0</version>
</dependency>
http://www.jgraph.com

This is a fairly simple graph construction and traversal problem. You do not need any libraries. You can do it in a simple java class. For e.g.
http://it-essence.xs4all.nl/roller/technology/entry/three_tree_traversals_in_java

It sounds like you'd want to implement nodes as class instances and links as references. Using maps to implement graph edges would be quite complicated and inefficient. Little wonder you'd want to clean up your code. I'm not sure I understand your problem perfectly, but this ought to be close:
// Null nodes are the simplest type. They represent missing children.
class NullNode {
// Get the values of all leaves descended from this node as a set.
Set<Integer> getValues() { return new HashSet(0); }
// Get the values descended from the node at the end of the given
// key path as a set. For a null node, this should not be called.
Set<Integer> getValues(int [] path, int i) { raise new IllegalOperationException(); }
// Initiate the search for values. The only way that's okay
// for null nodes is when the path is empty.
Set<Integer> getValues(int [] path) {
if (path.length == 0)
return new HashSet(0);
else
raise new IllegalOperationException();
}
}
// A regular node is a null node with a key. It should
// never be instantiated. Use Interior or Leaf nodes for that.
abstract class Node extends NullNode {
int key;
// Initiate the search for values. Only descend if the key matches.
Set<Integer> getValues(int [] path) {
return (path.length > 0 && path[0] == key) ? getValues(path, 1) : new HashSet(0);
}
}
// Interior nodes have two children, which may be Null, Interior, or Leaf.
class InteriorNode extends Node {
Node left, right;
Set<Integer> getValues() {
Set<Integer> v = left.getValues();
v.addAll(right.getValues());
return v;
}
Set<Integer> getValues(int [] path, int i) {
if (i + 1 < path.length) {
// Again we only descend if the key matches.
if (path[i + 1] == left.key) return getValues(left, i + 1);
if (path[i + 1] == right.key) return getValues(right, i + 1);
return new HashSet(0);
}
return getValues(); // Get values from both children.
}
}
// A leaf node has no children and a value.
class LeafNode extends Node {
int value;
Set<Integer> getValues() {
HashSet<Integer> v = new HashSet(1);
v.add(value);
return v;
}
Set<Integer> getValues(int [] path, int i) {
return (i + 1 >= path.length) ? getValues() : new HashSet(0);
}
}

The best graph library which I have found is not written in Java but in Scala and makes usage of some powerful scala features not available in Java, such as abstract types.
It is called Graph for Scala and it is extremely comprehensive, but I have to warn you that while Scala and Java they are intercompatible (you can build them in the same project and call a Java class from a Scala class and vice-versa), some problems might rise when calling Scala from Java when it comes to some features which are not available in Java.
http://www.assembla.com/spaces/scala-graph/wiki

Is there any custom Map implementation or third-party library for solve my situation for cleanup my code from manipulation manually such as String.split or loop and condition statements?
If you want to remove the freedom to written manipulate code then you can create your own libraries. You can easily create libraries in Eclipse by exporting your classes into a Jar file, which I would presume is a trivial task in NetBeans.
If you want to protect against changes to the graph after construction then you need to create an immutable data structure. With an immutable graph structure you have to view the Graph as a Universe, and each operation is a GraphOperation. You can never modify a Graph, only create a new Graph that results from crossing the Graph with your list of GraphOperations. Presuming your Graph structure holds unique node values, this will not pose too much of a problem, since you can happily describe relations using values. Your code will look something like this:
Graph graph2 = graph1.process(graph1.GetTopNode().RemoveLeft());
graph2 = graph2.process(graph2.GetNode(7).AddRight(8));
GetTopNode() returns an object that only provides a view of the nodes. RemoveLeft() returns a GraphOperation object, which Graph.process() uses to create a new graph from the operation. If you want, it could just return a Graph implementation that internally stores a link to graph1, and the list of GraphOperation instances that have been passed into it, allowing you to avoid copying the graph structures too often (pretty much like a string buffer).

If you are looking for Graph database and manipulation in Java, Neo4j might help you. This can be more than what you have bargained for if you are looking for a perfect Graph DB and manipulation API.
It gives you very advanced options to traverse the graph nodes, relationships, auditing. Neo4j is being used across organizations to store very complex hierarchical data, the performance by Neo4j is far better than oracle based R-DB's for complex hierarchical databases.

How do you query object collections in Java (Criteria/SQL-like)?

Suppose you have a collection of a few hundred in-memory objects and you need to query this List to return objects matching some SQL or Criteria like query. For example, you might have a List of Car objects and you want to return all cars made during the 1960s, with a license plate that starts with AZ, ordered by the name of the car model.
I know about JoSQL, has anyone used this, or have any experience with other/homegrown solutions?

Filtering is one way to do this, as discussed in other answers.
Filtering is not scalable though. On the surface time complexity would appear to be O(n) (i.e. already not scalable if the number of objects in the collection will grow), but actually because one or more tests need to be applied to each object depending on the query, time complexity more accurately is O(n t) where t is the number of tests to apply to each object.
So performance will degrade as additional objects are added to the collection, and/or as the number of tests in the query increases.
There is another way to do this, using indexing and set theory.
One approach is to build indexes on the fields within the objects stored in your collection and which you will subsequently test in your query.
Say you have a collection of Car objects and every Car object has a field color. Say your query is the equivalent of "SELECT * FROM cars WHERE Car.color = 'blue'". You could build an index on Car.color, which would basically look like this:
'blue' -> {Car{name=blue_car_1, color='blue'}, Car{name=blue_car_2, color='blue'}}
'red' -> {Car{name=red_car_1, color='red'}, Car{name=red_car_2, color='red'}}
Then given a query WHERE Car.color = 'blue', the set of blue cars could be retrieved in O(1) time complexity. If there were additional tests in your query, you could then test each car in that candidate set to check if it matched the remaining tests in your query. Since the candidate set is likely to be significantly smaller than the entire collection, time complexity is less than O(n) (in the engineering sense, see comments below). Performance does not degrade as much, when additional objects are added to the collection. But this is still not perfect, read on.
Another approach, is what I would refer to as a standing query index. To explain: with conventional iteration and filtering, the collection is iterated and every object is tested to see if it matches the query. So filtering is like running a query over a collection. A standing query index would be the other way around, where the collection is instead run over the query, but only once for each object in the collection, even though the collection could be queried any number of times.
A standing query index would be like registering a query with some sort of intelligent collection, such that as objects are added to and removed from the collection, the collection would automatically test each object against all of the standing queries which have been registered with it. If an object matches a standing query then the collection could add/remove it to/from a set dedicated to storing objects matching that query. Subsequently, objects matching any of the registered queries could be retrieved in O(1) time complexity.
The information above is taken from CQEngine (Collection Query Engine). This basically is a NoSQL query engine for retrieving objects from Java collections using SQL-like queries, without the overhead of iterating through the collection. It is built around the ideas above, plus some more. Disclaimer: I am the author. It's open source and in maven central. If you find it helpful please upvote this answer!

I have used Apache Commons JXPath in a production application. It allows you to apply XPath expressions to graphs of objects in Java.

yes, I know it's an old post, but technologies appear everyday and the answer will change in the time.
I think this is a good problem to solve it with LambdaJ. You can find it here:
http://code.google.com/p/lambdaj/
Here you have an example:
LOOK FOR ACTIVE CUSTOMERS // (Iterable version)
List<Customer> activeCustomers = new ArrayList<Customer>();
for (Customer customer : customers) {
if (customer.isActive()) {
activeCusomers.add(customer);
}
}
LambdaJ version
List<Customer> activeCustomers = select(customers,
having(on(Customer.class).isActive()));
Of course, having this kind of beauty impacts in the performance (a little... an average of 2 times), but can you find a more readable code?
It has many many features, another example could be sorting:
Sort Iterative
List<Person> sortedByAgePersons = new ArrayList<Person>(persons);
Collections.sort(sortedByAgePersons, new Comparator<Person>() {
public int compare(Person p1, Person p2) {
return Integer.valueOf(p1.getAge()).compareTo(p2.getAge());
}
});
Sort with lambda
List<Person> sortedByAgePersons = sort(persons, on(Person.class).getAge());
Update: after java 8 you can use out of the box lambda expressions, like:
List<Customer> activeCustomers = customers.stream()
.filter(Customer::isActive)
.collect(Collectors.toList());

Continuing the Comparator theme, you may also want to take a look at the Google Collections API. In particular, they have an interface called Predicate, which serves a similar role to Comparator, in that it is a simple interface that can be used by a filtering method, like Sets.filter. They include a whole bunch of composite predicate implementations, to do ANDs, ORs, etc.
Depending on the size of your data set, it may make more sense to use this approach than a SQL or external relational database approach.

If you need a single concrete match, you can have the class implement Comparator, then create a standalone object with all the hashed fields included and use it to return the index of the match. When you want to find more than one (potentially) object in the collection, you'll have to turn to a library like JoSQL (which has worked well in the trivial cases I've used it for).
In general, I tend to embed Derby into even my small applications, use Hibernate annotations to define my model classes and let Hibernate deal with caching schemes to keep everything fast.

I would use a Comparator that takes a range of years and license plate pattern as input parameters. Then just iterate through your collection and copy the objects that match. You'd likely end up making a whole package of custom Comparators with this approach.

The Comparator option is not bad, especially if you use anonymous classes (so as not to create redundant classes in the project), but eventually when you look at the flow of comparisons, it's pretty much just like looping over the entire collection yourself, specifying exactly the conditions for matching items:
if (Car car : cars) {
if (1959 < car.getYear() && 1970 > car.getYear() &&
car.getLicense().startsWith("AZ")) {
result.add(car);
}
}
Then there's the sorting... that might be a pain in the backside, but luckily there's class Collections and its sort methods, one of which receives a Comparator...