Develop Reference

Different result on running Flink in local mode and Yarn cluster

I run a code using Flink Java API that gets some bytes from Kafka and parses it following by inserting into Cassandra database using another library static method (both parsing and inserting results is done by the library). Running code on local in IDE, I get the desired answer, but running on YARN cluster the parse method didn't work as expected!
public class Test {
static HashMap<Integer, Object> ConfigHashMap = new HashMap<>();
public static void main(String[] args) throws Exception {
CassandraConnection.connect();
Parser.setInsert(true);
stream.flatMap(new FlatMapFunction<byte[], Void>() {
#Override
public void flatMap(byte[] value, Collector<Void> out) throws Exception {
Parser.parse(ByteBuffer.wrap(value), ConfigHashMap);
// Parser.parse(ByteBuffer.wrap(value));
}
});
env.execute();
}
}
There is a static HashMap field in the Parser class that configuration of parsing data is based on its information, and data will insert it during the execution. The problem running on YARN was this data was not available for taskmanagers and they just print config is not available!
So I redefine that HashMap as a parameter for parse method, but no differences in results!
How can I fix the problem?

I changed static methods and fields to non-static and using RichFlatMapFunction solved the problem.
stream.flatMap(new RichFlatMapFunction<byte[], Void>() {
CassandraConnection con = new CassandraConnection();
int i = 0 ;
#Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
con.connect();
}
#Override
public void flatMap(byte[] value, Collector<Void> out) throws Exception {
ByteBuffer tb = ByteBuffer.wrap(value);
np.parse(tb, ConfigHashMap, con);
}
});

Error when trying to get access to my remote client from my remote EJB

I've been stuck for two days now, trying to resolve a communication problem between my singleton session bean and a standalone (rmi) client.
Here is the point: I have a remote session bean (LoadBalancer) which can be accessed by a client (ChatRoom), and which can also access to the client... in theory.
The remote interface:
public interface ILoadBalancer{
public void addChatRoom( IChatRoom chatRoom );
public void removeChatRoom( IChatRoom chatRoom );
}
The session bean:
#Remote(ILoadBalancer.class)
#Singleton
public class LoadBalancer implements ILoadBalancer {
/**
* List of available chat rooms
*/
private Map<String, IChatRoom> chatRooms;
private static final Logger logger = Logger.getLogger("loadBalancerLogger");
#PostConstruct
public void init() {
chatRooms = new HashMap<String, IChatRoom>();
}
#Override
public void addChatRoom(IChatRoom chatRoom) {
// Adding a new chat room to the list of available chat rooms
try {
chatRooms.put(chatRoom.getName(), chatRoom);
logger.log(Level.INFO, "Enregistrement d'un salon de discussion {0}", chatRoom.getName());
} catch (RemoteException ex) {
Logger.getLogger(LoadBalancer.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Override
public void removeChatRoom(IChatRoom chatRoom) {
// Removing a chat room from the list of available chat rooms
try {
chatRooms.remove(chatRoom.getName());
logger.log(Level.INFO, "Suppression du salon de discussion {0}", chatRoom.getName());
} catch (RemoteException ex) {
Logger.getLogger(LoadBalancer.class.getName()).log(Level.SEVERE, null, ex);
}
}
These are deployed in a glassfish server (4.1).
And now the standalone client using RMI:
public interface IChatRoom extends Remote {
public int postMessage( String pseudo, String message) throws RemoteException;
public int subscribe( String pseudo) throws RemoteException;
public List<String> getNewMessages(String pseudo, int sequenceNumber) throws RemoteException;
public String getName() throws RemoteException;
}
And the class implementing it:
public class ChatRoom implements IChatRoom, Serializable {
private String name;
private List<String> users;
private List<String> messages;
private static final Logger logger = Logger.getLogger("chatRoomLogger");
/**
* A reference to the load balancer
*/
private ILoadBalancer loadBalancer;
public ChatRoom(String name) throws RemoteException, NamingException {
this.name = name;
users = new ArrayList<String>();
messages = new ArrayList<String>();
// Making the chat room available via RMI
UnicastRemoteObject.exportObject(this);
// Retrieving a reference to the loadBalancer
Properties props = new Properties();
props.setProperty(Context.INITIAL_CONTEXT_FACTORY, "com.sun.enterprise.naming.impl.SerialInitContextFactory");
props.setProperty(Context.STATE_FACTORIES, "com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl");
props.setProperty(Context.URL_PKG_PREFIXES, "com.sun.enterprise.naming");
props.setProperty("org.omg.CORBA.ORBInitialHost", "localhost");
props.setProperty("org.omg.CORBA.ORBInitialPort", "3700");
Context ctx = new InitialContext(props);
loadBalancer = (ILoadBalancer) ctx.lookup("java:global/loadBalancer/LoadBalancer!sn.esp.dgi.core.ILoadBalancer");
loadBalancer.addChatRoom(this);
}
#Override
public int postMessage(String pseudo, String message) throws RemoteException {
// Not relevant to the problem
}
#Override
public int subscribe(String pseudo) throws RemoteException {
// Not relevant to the problem
}
#Override
public List<String> getNewMessages(String pseudo, int sequenceNumber) throws RemoteException {
// Not relevant to the problem
}
#Override
public String getName() throws RemoteException {
// Not Relevant to the problem
}
}
As you have seen, the client has a remote reference to the session bean, and the session bean has also a remote reference to all the clients that have subscribed to it.
The client is launched with this code:
public class ChatRoomLauncher {
public static void main( String[] args ) {
if( args.length < 1 ) {
System.out.println("Usage: java ChatRoomLauncher <nomChatRoom>");
System.exit(-1);
}
String chatRoomName = args[0];
try {
IChatRoom chatRoom = new ChatRoom(chatRoomName);
System.out.println("Salon de discussion "+ chatRoomName +" démarré ");
} catch (RemoteException ex) {
Logger.getLogger(ChatRoomLauncher.class.getName()).log(Level.SEVERE, null, ex);
} catch (NamingException ex) {
Logger.getLogger(ChatRoomLauncher.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
After generating the stub and skeleton of respectively IChatRoom and ChatRoom, and placing the stub of IChatRoom in the directory where is deployed the LoadBalancer, I get the following exceptions which are my nightmares since two days:
Exception in thread "main" javax.ejb.NoSuchEJBException
at sn.esp.dgi.core._ILoadBalancer_Wrapper.addChatRoom(sn/esp/dgi/core/_ILoadBalancer_Wrapper.java)
at sn.esp.dgi.server.ChatRoom.<init>(ChatRoom.java:76)
at sn.esp.dgi.server.ChatRoomLauncher.main(ChatRoomLauncher.java:29)
Caused by: java.rmi.NoSuchObjectException: CORBA INV_OBJREF 1330447539 No; nested exception is:
org.omg.CORBA.INV_OBJREF: WARNING: 00511203: Class sn.esp.dgi.server.ChatRoom not exported, or else is actually a JRMP stub vmcid: OMG minor code: 1203 completed: No
at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.mapSystemException(Util.java:260)
at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:211)
at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:150)
at com.sun.corba.ee.impl.presentation.rmi.codegen.CodegenStubBase.invoke(CodegenStubBase.java:226)
at sn.esp.dgi.core.__ILoadBalancer_Remote_DynamicStub.addChatRoom(sn/esp/dgi/core/__ILoadBalancer_Remote_DynamicStub.java)
... 3 more
Caused by: org.omg.CORBA.INV_OBJREF: WARNING: 00511203: Class sn.esp.dgi.server.ChatRoom not exported, or else is actually a JRMP stub vmcid: OMG minor code: 1203 completed: No
at com.sun.proxy.$Proxy54.objectNotExported(Unknown Source)
at com.sun.corba.ee.impl.util.Utility.autoConnect(Utility.java:160)
at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.writeRemoteObject(Util.java:473)
at com.sun.corba.ee.impl.presentation.rmi.DynamicMethodMarshallerImpl$12.write(DynamicMethodMarshallerImpl.java:356)
at com.sun.corba.ee.impl.presentation.rmi.DynamicMethodMarshallerImpl.writeArguments(DynamicMethodMarshallerImpl.java:465)
at com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:196)
... 6 more
The exception tells me that I have not exported the chatRoom, though I have generated and placed its stub in the directory where I think it should be.
I've tried to change the instruction UnicastRemoteObject.exportObject(this) by PortableRemoteObject.exportObject(this), but I still get an error, different from this one.
Any help would be greatly appreciated.
Thanks

I had the exact same issues (EJB running on Glassfish 4 server, client with RMI callback object) and finally found a solution! The tricks were to:
Make the client implementation inherit from PortableRemoteObject, i.e. in your example:
public class ChatRoom extends PortableRemoteObject implements IChatRoom, Serializable
Note:
There were 3 choices for PortableRemoteObject offered by Eclipse:
javax.rmi.PortableRemoteObject -> did not work, wanted stubs which I did not generate for the client object (error message: java.rmi.StubNotFoundException: Stub class not found)
com.sun.corba.se.impl.javax.rmi.PortableRemoteObject -> same issue
com.sun.corba.ee.impl.javax.rmi.PortableRemoteObject -> WORKED, without generating stubs!!
Export the object in the constructor (in the same place where you have UnicastRemoteObject.exportObject(this);). Extending the class from com.sun.corba.ee.impl.javax.rmi.PortableRemoteObject allowed simply to use:
exportObject(this); // i.e. non-static reference is required
Note: if extending from javax.rmi.PortableRemoteObject then static reference to PortableRemoteObject.exportObject(this) is needed but then one needs the stub as I've found. You could try that, too, since you have the stubs.
Hope this helps!

Server breaks on Naming.rebind

I'm doing an agenda on server and everything was going fine, I started the RMI registry and the server was working fine, I test the code and I was able to login.
now I want to start putting the codes in methods and calling them in the client from the server,and when I put the Naming.rebind("server", i); in the server to call methods the code breaks.
My project is contain three classes: server, client and interface that calls the methods from the server to the client. This is the code of the server:
public class ServerAgendas extends UnicastRemoteObject
implements InterfaceServer {
private static final long serialVersionUID = 1L;
protected ServerAgendas() throws RemoteException {
super();
// TODO Auto-generated constructor stub
}
static String Clientusername;
static String Clientpassword;
public static void main (String args[]) throws NotBoundException,
IOException {
System.out.println("1");
try{
InterfaceServer i = new ServerAgendas();
System.out.println("2");
Naming.rebind("server",i);
System.out.println("Serveur agendas pret");
}
catch (Exception e){
System.err.println("Error "+e.getMessage());
}
ServerSocket ASocket = new ServerSocket(portNumber);
Socket connectionSocket = ASocket.accept();
}
public boolean login(String ClientUsername, String ClientPassword)
throws IOException, SQLException {
}
in the client side I use this to call the methods:
InterfaceServer i = (InterfaceServer)Naming.lookup("server");
System.out.println(i.login(username, password));
and the interface contain:
public interface InterfaceServer extends Remote{
public boolean login(String Clientusername, String Clientpassword)
throws RemoteException, IOException, SQLException;
I run the registry at first then I run the server, the console shows:
1
2
Error RemoteException occurred in server thread; nested exception is:
java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
java.lang.ClassNotFoundException: InterfaceServer
I used the 1 and 2 to see where it is breaking, it is breaking on:
Naming.rebind("server",i);
What do I need to do to avoid this exception occurring?

The class mentioned in the message isn't available to the Registry on its CLASSPATH.
The simplest solution to this and several other problems is to run the Registry inside the server JVM, with LocateRegistry.createRegistry().
Otherwise you will have to figure out how to start the rmiregistry command with a CLASSPATH argument, or else use the codebase feature, which is another kettle of fish altogether.

neo4j rest graphdb hangs when connecting to remote heroku instance

public class Test
{
private static RestAPI rest = new RestAPIFacade("myIp","username","password");
public static void main(String[] args)
{
Map<String, Object> foo = new HashMap<String, Object>();
foo.put("Test key", "testing");
rest.createNode(foo);
}
}
No output it just hangs on connection indefinitely.
Environment:
Eclipse
JDK 7
neo4j-rest-binding 1.9: https://github.com/neo4j/java-rest-binding
Heroku
Any ideas as to why this just hangs?
The following code works:
public class Test
{
private static RestAPI rest = new RestAPIFacade("myIp","username","password");
public static void main(String[] args)
{
Node node = rest.getNodeById(1);
}
}
So it stands that I can correctly retrieve remote values.

I guess this is caused by lacking usage of transactions. By default neo4j-rest-binding aggregates multiple operations into one request (aka one transaction). There are 2 ways to deal with this:
change transactional behaviour to "1 operation = 1 transaction" by setting
-Dorg.neo4j.rest.batch_transaction=false for your JVM. Be aware this could impact performance since every atomic operation is a seperate REST request.
use transactions in your code:
.
RestGraphDatabse db = new RestGraphDatabase("http://localhost:7474/db/data",username,password);
Transaction tx = db.beginTx();
try {
Node node = db.createNode();
node.setPropery("key", "value");
tx.success();
} finally {
tx.finish();
}

Detecting when a handler couldn't be started when embedding Jetty

I'm embedding Jetty in a similar manner as described here. When the RequestLogHandler can't open the specified logfile, it throws an exception which is unfortunately caught by org.eclipse.jetty.server.Server and swallowed (but logged first, at least). This means that there's no obvious way for me to tell if the log handler was started correctly.
Is there a way that I'm missing to detect when a handler couldn't start?

This idea is based on the implementation of WebAppContext where you can use WebAppContext.getUnavailableException() to determine whether the context was initialized successfully.
Simply replace the default implementation of Server and Context with your own:
public static class MyContext extends Context {
private Exception _exception;
#Override
protected void doStart() throws Exception {
try {
super.doStart();
} catch (final Exception e) {
_exception = e;
}
}
#Override
protected void doStop() throws Exception {
try {
super.doStop();
} finally {
_exception = null;
}
}
public Exception getException() {
return _exception;
}
}
public static class MyServer extends Server implements InitializingBean {
public void afterPropertiesSet() throws Exception {
start();
for (final Handler h : getHandlers()) {
if (h instanceof MyContext) {
final MyContext c = (MyContext) h;
if (c.getException() != null) {
throw new RuntimeException("failed to init context " + c.getDisplayName(),
c.getException());
}
}
}
}
}
In your beans.xml, simply replace org.mortbay.jetty.Server (and remove init-method="start") and org.mortbay.jetty.servlet.Context with your own implementations.
This code is for Jetty 6 though (as is the example you linked to), as that's what I have around. I didn't test it though, but it's pretty much the same as we are successfully using in conjunction with WebAppContext. In order to extend this to RequestLogHandler, you could either do the same for just any handler you are using or create a decorator to wrap any handler. You may want to look at org.mortbay.jetty.handler.HandlerWrapper for this purpose.

How about modifying the jetty code? You could add some simple println statements in strategic places in the RequestLogHandler which would indicate to you whether or not the handler was started.