public class Test
{
private static RestAPI rest = new RestAPIFacade("myIp","username","password");
public static void main(String[] args)
{
Map<String, Object> foo = new HashMap<String, Object>();
foo.put("Test key", "testing");
rest.createNode(foo);
}
}
No output it just hangs on connection indefinitely.
Environment:
Eclipse
JDK 7
neo4j-rest-binding 1.9: https://github.com/neo4j/java-rest-binding
Heroku
Any ideas as to why this just hangs?
The following code works:
public class Test
{
private static RestAPI rest = new RestAPIFacade("myIp","username","password");
public static void main(String[] args)
{
Node node = rest.getNodeById(1);
}
}
So it stands that I can correctly retrieve remote values.
I guess this is caused by lacking usage of transactions. By default neo4j-rest-binding aggregates multiple operations into one request (aka one transaction). There are 2 ways to deal with this:
change transactional behaviour to "1 operation = 1 transaction" by setting
-Dorg.neo4j.rest.batch_transaction=false for your JVM. Be aware this could impact performance since every atomic operation is a seperate REST request.
use transactions in your code:
.
RestGraphDatabse db = new RestGraphDatabase("http://localhost:7474/db/data",username,password);
Transaction tx = db.beginTx();
try {
Node node = db.createNode();
node.setPropery("key", "value");
tx.success();
} finally {
tx.finish();
}
Related
I have a SpringBoot gradle project using apache flink to process datastream signals. When a new signal comes through the datastream, I would like to query look up (i.e. findById() ) it's details using an ID in a postgres database table which is already created in order to get additional information about the signal and enrich the data. I would like to avoid using spring dependencies to perform the lookup (i.e Autowire repository) and want to stick with flink implementation for the lookup.
Where can i specify how to add the postgres connection config information such as port, database, url, username, password etc... (for simplicity purposes can assume the postgres db is local in my machine). Is it as simple as adding the configuration to the application.properties file? if so how can i write the query method to look up the record in the postgres table when searching by non primary key value?
Some online sources are suggesting using this skeleton code but I am not sure how/id it fits my use case. (I have a EventEntity model created which contains all the params/columns from the table which i'm looking up).
like so
public class DatabaseMapper extends RichFlatMapFunction<String, EventEntity> {
// Declare DB connection & query statements
public void open(Configuration parameters) throws Exception {
//Initialize DB connection
//prepare query statements
}
#Override
public void flatMap(String value, Collector<EventEntity> out) throws Exception {
}
}
Your sample code is correct. You can set all your custom initialization and preparation code for PostgreSQL in open() method. Then you can use your pre-configured fields in your flatMap() function.
Here is one sample for Redis operations
I have used RichAsyncFunction here and I suggest you do the same as it is suggested as best practice. Read here for more: https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/operators/asyncio.html)
You can pass configuration parameteres in your constructor method and use it in your initialization process
public static class AsyncRedisOperations extends RichAsyncFunction<Object,Object> {
private JedisPool jedisPool;
private Configuration redisConf;
public AsyncRedisOperations(Configuration redisConf) {
this.redisConf = redisConf;
}
#Override
public void open(Configuration parameters) {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(this.redisConf.getInteger("pool", 8));
jedisPoolConfig.setMaxIdle(this.redisConf.getInteger("pool", 8));
jedisPoolConfig.setMaxWaitMillis(this.redisConf.getInteger("maxWait", 0));
JedisPool jedisPool = new JedisPool(jedisPoolConfig,
this.redisConf.getString("host", "192.168.10.10"),
this.redisConf.getInteger("port", 6379), 5000);
try {
this.jedisPool = jedisPool;
this.logger.info("Redis connected: " + jedisPool.getResource().isConnected());
} catch (Exception e) {
this.logger.error(BaseUtil.append("Exception while connecting Redis"));
}
}
#Override
public void asyncInvoke(Object in, ResultFuture<Object> out) {
try (Jedis jedis = this.jedisPool.getResource()) {
String key = jedis.get(key);
this.logger.info("Redis Key: " + key);
}
}
}
I run a code using Flink Java API that gets some bytes from Kafka and parses it following by inserting into Cassandra database using another library static method (both parsing and inserting results is done by the library). Running code on local in IDE, I get the desired answer, but running on YARN cluster the parse method didn't work as expected!
public class Test {
static HashMap<Integer, Object> ConfigHashMap = new HashMap<>();
public static void main(String[] args) throws Exception {
CassandraConnection.connect();
Parser.setInsert(true);
stream.flatMap(new FlatMapFunction<byte[], Void>() {
#Override
public void flatMap(byte[] value, Collector<Void> out) throws Exception {
Parser.parse(ByteBuffer.wrap(value), ConfigHashMap);
// Parser.parse(ByteBuffer.wrap(value));
}
});
env.execute();
}
}
There is a static HashMap field in the Parser class that configuration of parsing data is based on its information, and data will insert it during the execution. The problem running on YARN was this data was not available for taskmanagers and they just print config is not available!
So I redefine that HashMap as a parameter for parse method, but no differences in results!
How can I fix the problem?
I changed static methods and fields to non-static and using RichFlatMapFunction solved the problem.
stream.flatMap(new RichFlatMapFunction<byte[], Void>() {
CassandraConnection con = new CassandraConnection();
int i = 0 ;
#Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
con.connect();
}
#Override
public void flatMap(byte[] value, Collector<Void> out) throws Exception {
ByteBuffer tb = ByteBuffer.wrap(value);
np.parse(tb, ConfigHashMap, con);
}
});
I'm trying to run the following example but I did not succeed.
The intention is sharing objects through JNDI between two Java programs running on different virtual machines. I'm using TomEE 1.5.1 and TomEE 1.6
The JNDI parameters that I'm using are:
Hashtable<String, String> ctxProps = new Hashtable<String, String>();
ctxProps.put("java.naming.factory.initial","org.apache.openejb.client.RemoteInitialContextFactory");
ctxProps.put("java.naming.provider.url", "http://myjndihost.com:8080/tomee/ejb");
ctxProps.put("java.naming.security.principal", "tomee");
ctxProps.put("java.naming.security.credentials", "tomee");
The Sharing class provide 3 methods: connect (called within the constructor), set and get, and It's source code is here:
public class Sharing
{
private Context context=null;
public Sharing() throws Exception
{
connect();
}
private void connect() throws Exception
{
Hashtable<String, String> ctxProps = new Hashtable<String, String>();
ctxProps.put("java.naming.factory.initial","org.apache.openejb.client.RemoteInitialContextFactory");
ctxProps.put("java.naming.provider.url", "http://localhost:8080/tomee/ejb");
ctxProps.put("java.naming.security.principal", "tomee");
ctxProps.put("java.naming.security.credentials", "tomee");
context=new InitialContext(ctxProps);
context=context.createSubcontext("DEMO");
}
public void set(String key, Serializable value) throws Exception
{
try
{
context.bind(key,value);
}
catch(NameAlreadyBoundException ex)
{
context.rebind(key,value);
}
}
public Object get(String key) throws Exception
{
try
{
return context.lookup(key);
}
catch(NameNotFoundException e)
{
return null;
}
}
}
Then, I run this TestSet.java
public class TestSet
{
public static void main(String[] args) throws Exception
{
Sharing share = new Sharing();
share.put("fecha", new Date());
}
}
It's throught an javax.naming.OperationNotSupportedException:
Exception in thread "main" javax.naming.OperationNotSupportedException
at org.apache.openejb.client.JNDIContext.createSubcontext(JNDIContext.java:551)
at javax.naming.InitialContext.createSubcontext(InitialContext.java:483)
at demo.Sharing.connect(Sharing.java:32)
at demo.Sharing.<init>(Sharing.java:20)
at demo.TestSet.main(TestSet.java:9)
But, if I remove the createSubcontect call (in line 32 of Sharing.java) then, the next exception is in the line 39, when
try to bind an object.
Exception in thread "main" javax.naming.OperationNotSupportedException
at org.apache.openejb.client.JNDIContext.bind(JNDIContext.java:511)
at javax.naming.InitialContext.bind(InitialContext.java:419)
at demo.Sharing.put(Sharing.java:39)
at demo.TestSet.main(TestSet.java:10)
Of course, next step will be run TestGet.java but the time has not arrived because
I could not run succefully TestSet yet.
The original example is taken from here, using WebLogic.
http://www.javaworld.com/article/2076440/jndi/use-jndi-to-share-objects-between-different-virtual-machines.html
Thanks a lot.
Pablo.
I'm new to graph db and i'm having problems to get the api work within a transaction.
I have a simple code that uses the neo4j graph db api to create nodes and relationship. My code runs in JUnit and tries to create 2 nodes and a relationship between them using begin and end transaction given below.
The code works fine in a happy scenario. However, if something fails within the code, the nodes are still committed into the graph database. Not sure if i'm doing something wrong out here. I would have expected the 2 nodes created to be rolled back.
Here is the code snippet:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { "classpath:/applicationContext.xml" })
public class RestBatchLoaderTest {
#Autowired
SpringRestGraphDatabase graphDatabaseService;
#Test
public void createNode() {
Transaction tx =graphDatabaseService.beginTx();
try {
Map<String,Object> nodeprops1 = new HashMap<String, Object>();
nodeprops1.put("name", "James Parker");
nodeprops1.put("age", Integer.valueOf(11));
Node james = graphDatabaseService.createNode(nodeprops1);
Assert.assertNotNull(james);
Map<String,Object> nodeprops2 = new HashMap<String, Object>();
nodeprops2.put("name", "Bing P");
nodeprops2.put("age", Integer.valueOf(34));
Node bing= graphDatabaseService.createNode(nodeprops2);
Node aa = null;
// Failure point: should rollback the previous node in the finally.
graphDatabaseService.remove(aa);
Map<String,Object> relprops = new HashMap<String, Object>();
RelationshipType type = new RelationshipType() {
#Override
public String name() {
return "MARRIED_TO";
}
};
graphDatabaseService.createRelationship(joe, jane, type, relprops);
tx.success();
} finally {
tx.finish();
}
}
The graphDatabaseService object is autowired using spring configuration. Here is the configuration:
<neo4j:config graphDatabaseService="graphDatabaseService"/>
<bean id="graphDatabaseService" class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase">
<constructor-arg value="http://localhost:7474/db/data/"/>
</bean>
Also, I notice tx object is an instance of NullTransaction when graphDatabaseService.beginTx() is called in the code above.
Any ideas, what is going wrong?
Thanks.
I think figured out what the problem was. The configuration needs to have batch enabled - true. Also i used the RestAPI wrapper to the graph database object to run it as one atomic code. See code below:
#Autowired
SpringRestGraphDatabase graphDatabaseService;
private RestAPI restAPI;
#Before
public void init(){
this.restAPI = ((RestGraphDatabase)graphDatabaseService).getRestAPI();
}
#Test
public void testEnableBatchTransactions() throws Exception {
System.setProperty(Config.CONFIG_BATCH_TRANSACTION,"true");
Transaction tx = restAPI.beginTx();
try {
Node n1 = restAPI.createNode(map("name", "node1"));
Node n2 = restAPI.createNode(map("name", "node2"));
Node n3 = restAPI.createNode(map("name", "node3"));
//String s = null;
//s.toString();
Node n4 = restAPI.createNode(map("name", "node4"));
tx.success();
} finally {
tx.finish();
}
assertTrue(tx instanceof BatchTransaction);
}
Also System.setProperty(Config.CONFIG_BATCH_TRANSACTION,"true"); enables the batch mode.
To test this, try un-commenting the code snippet and run the test. Nodes n1, n2 and n3 will not be committed in the db.
You specified graphDatabaseService.remove(aa); as your failure point as aa is NULL. Looking into the documentation of org.springframework.data.neo4j.rest.SpringRestGraphDatabase there is no Exception documented that is being thrown if the node is NULL. Have you verified that an exception is actually thrown? Otherwise your code will run through to tx.success();. If an exception is thrown, please specify further what version of neo4j and spring you are using.
Edit:
After reading a little more, I see in the source of org.springframework.data.neo4j.rest.SpringRestGraphDatabase that it should give you a NullTransaction that basically does nothing (see here).
Furthermore, the Spring Data neo4j documentationstates that each operation is in its own transaction as the neo4j REST adapter does not allow transaction that span over multiple operations (see here).
I'm working on a Neo4J database, using it in a little social network. The idea is:
As a user, you connect to the website, and it connects you to a N4J db.
I tried to avoid problem related to multiple instances of the db by creating it through a Singleton. However, I think my users must be able to "connect" and "disconnect", so I created method for that, but when I use disconnect() on a user, the others run through a NullPointerException due to the fact that they can't access the db anymore (which I thought was handled by the Singleton).
Here is the code, which I believe will make things clearer :
The main that I use to test my code :
public static void main(String[] args) {
Node root;
N4JAdmin admin = new N4JAdmin();
Transaction tx = GraphDB.getGraphDb().beginTx();
try {
root = GraphDB.getGraphDb().createNode();
root.setProperty("Nom", "aname");
root.setProperty("Prenom", "afirstname");
tx.success();
} finally {
tx.finish();
}
N4JUser sr = admin.addUser("Ribeiro", "Swen", 1);
//14 more addUser, it creates nodes in the db with the 3 properties, works fine
// The following connect() calls are commented because the errors occurs with or without them
// sr.connect();
// the 14 others...
sr.addAcquaintance(cw.getUserNode());
sr.addAcquaintance(fc.getUserNode());
sr.disconnect();
System.out.println("+++++++++ sr disconnected ++++++++++");
bm.addAcquaintance(cm.getUserNode());
// and more operations...
GraphDB.getGraphDb().shutdown();
}
the addAquaintance code :
public void addAcquaintance(Node target) {
Transaction tx = GraphDB.getGraphDb().beginTx();
try {
this.userNode.createRelationshipTo(target, RelTypes.CONNAIT);
System.out.println(this.userNode.getProperty("Nom")+" "+this.userNode.getProperty("Prenom")+" est maintenant ami avec : "+ target.getProperty("Nom")+" "+target.getProperty("Prenom"));
tx.success();
} finally {
tx.finish();
}
}
And the GraphDB class, which is my Singleton :
public final class GraphDB {
private static final String DB_PATH = "/Workties/database/workties.db";
private static GraphDatabaseService graphDb;
private static boolean instanciated = false;
private GraphDB(){
GraphDB.graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH);
registerShutdownHook(graphDb);
GraphDB.instanciated = true;
}
private static void registerShutdownHook( final GraphDatabaseService graphDb ){
Runtime.getRuntime().addShutdownHook( new Thread(){
#Override
public void run(){
graphDb.shutdown();
}
} );
}
public static GraphDatabaseService getGraphDb(){
if (!GraphDB.instanciated) {
new GraphDB();
System.out.println("New connection : " +GraphDB.graphDb.toString());
}
return GraphDB.graphDb;
}
}
Note : the instanciated attribute was added later, but even without it it didn't work.
My problem comes from the fact that I thought this type of error wouldn't happen using a Singleton, so I'm a bit stuck...
Thanks in advance for your help !
EDIT : the disconnect method :
public void disconnect(){
GraphDB.getGraphDb().shutdown();
}
You should not have to call shutdown for every user- once the DB is instantiated for the first time, it is enough to last you for all your querying etc. Why do you want to connect and disconnect?
The nullpointer is because shutdown shuts down Neo4j but your singleton still thinks instanciated=true, so it returns a handle to a graphDb that isn't really there. Even without the instanciated variable which you added, the singleton maintains a reference to graphDb after it's shutdown.
Strongly advise against doing the shutdown at all unless your app is terminating.