I'm pretty puzzled with this issue. I have an Apache Thrift 0.9.0 client and server. The client code goes like this:
this.transport = new TSocket(this.server, this.port);
final TProtocol protocol = new TBinaryProtocol(this.transport);
this.client = new ZKProtoService.Client(protocol);
This works fine. However, if I try to wrap the transport in a TFramedTransport
this.transport = new TSocket(this.server, this.port);
final TProtocol protocol = new TBinaryProtocol(new TFramedTransport(this.transport));
this.client = new ZKProtoService.Client(protocol);
I get the following obscure (no explanation message whatsoever) exception in the client side. Server side shows no error.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at com.blablabla.android.core.device.proto.ProtoService$Client.recv_open(ProtoService.java:108)
at com.blablabla.android.core.device.proto.ProtoService$Client.open(ProtoService.java:95)
at com.blablabla.simpleprotoclient.proto.ProtoClient.initializeCommunication(ProtoClient.java:411)
at com.blablabla.simpleprotoclient.proto.ProtoClient.doWork(ProtoClient.java:269)
at com.blablabla.simpleprotoclient.proto.ProtoClient.run(ProtoClient.java:499)
at java.lang.Thread.run(Thread.java:724)
It also fails if I use TCompactProtocol instead of TBinaryProtocol.
In the server side I have extended TProcessor with my own class since I need to reuse existing service handler (the service server-side IFace implementation) for this client:
#Override
public boolean process(final TProtocol in, final TProtocol out)
throws TException {
final TTransport t = in.getTransport();
final TSocket socket = (TSocket) t;
socket.setTimeout(ProtoServer.SOCKET_TIMEOUT);
final String clientAddress = socket.getSocket().getInetAddress()
.getHostAddress();
final int clientPort = socket.getSocket().getPort();
final String clientRemote = clientAddress + ":" + clientPort;
ProtoService.Processor<ProtoServiceHandler> processor = PROCESSORS
.get(clientRemote);
if (processor == null) {
final ProtoServiceHandler handler = new ProtoServiceHandler(
clientRemote);
processor = new ProtoService.Processor<ProtoServiceHandler>(
handler);
PROCESSORS.put(clientRemote, processor);
HANDLERS.put(clientRemote, handler);
ProtoClientConnectionChecker.addNewConnection(clientRemote,
socket);
}
return processor.process(in, out);
}
And this is how I start the server side:
TServerTransport serverTransport = new TServerSocket(DEFAULT_CONTROL_PORT);
TServer server = new TThreadPoolServer(new TThreadPoolServer.Args(
serverTransport).processor(new ControlProcessor()));
Thread thControlServer = new Thread(new StartServer("Control", server));
thControlServer.start();
I have some questions:
Is it correct to reuse service handler instances or I shouldn't be doing this?
Why does it fail when I use TFramedTransport or TCompactProtocol? How to fix this?
Any help on this issue is welcome. Thanks in advance!
I was having the same problem and finally found the answer. It is possible to set the transport type on the server, though this is not clear from most tutorials and examples I've found on the web. Have a look at all of the methods of the TServer.Args class (or the args classes for other servers, which extend TServer.Args). There are methods inputTransportFactory and outputTransportFactory. You can use new TFramedTransport.Factory() as inputs to each of these methods to declare which transport the server should use. In scala:
val handler = new ServiceStatusHandler
val processor = new ServiceStatus.Processor(handler)
val serverTransport = new TServerSocket(9090)
val args = new TServer.Args(serverTransport)
.processor(processor)
.inputTransportFactory(new TFramedTransport.Factory)
.outputTransportFactory(new TFramedTransport.Factory)
val server = new TSimpleServer(args)
println("Starting the simple server...")
server.serve()
Note that if you are using a TAsyncClient, you have no choice about the transport that you use. You must use TNonblockingTransport, which has only one standard implementation, TNonblockingSocket, which internally wraps whatever protocol you are using in a framed transport. It doesn't actually wrap your chosen protocol in a TFramedTransport, but it does prepend the length of the frame to the content that it writes, and expects the server to prepend the length of the response as well. This wasn't documented anywhere I found, but if you look at the source code and experiment with different combinations, you will find that with TSimpleServer you must use TFramedTransport to get it to work with an async client.
By the way, it's also worth noting that the docs say that a TNonblockingServer must use TFramedTransport in the outermost later of the transport. However, the examples don't show this being set in TNonblockingServer.Args, yet you still find that you must use TFramedTransport on the client side to successfully execute an rpc on the server. This is because TNonblockingServer.Args has its input and output protocols set to TFramedTransport by default (you can see this using reflection to inspect the fields of the superclass hierarchy or in the source code for the constructor of AbstractNonblockingServerArgs -- you can override the input and output transports, but the server will likely fail for the reasons discussed in the documentation).
When the issue happens with framed, but it works without framed, then you have an incompatible protocol stack on both ends. Choose one of the following:
either modify the server code to use framed as well
or do not use framed on the client
A good rule of thumb is, to always use the exact same protocol/transport stack on both ends. In the particular case it blows up, because framed adds a four-byte header holding the size of the message that follows. If the server does not use framed, these additional four bytes sent by the client will be interpreted (wrongly) as part of the message.
Altough the sample code in that answer
TNonblockingServer in thrift crashes when TFramedTransport opens is for C++, adding framed on the server should be very similar with Java.
PS: Yes, it is perfectly ok to re-use your handler. A typical handler is a stateless thing.
Related
I have generated client and server.
I need to add one more field to message in client and read it in server.
I think about to decorate TProtocol on client side and TProcessor on server side. E.g.:
// Client:
TTransport transport = new TSocket("localhost", 8888);
transport.open();
TProtocol protocol = new DecoratedProtocol(new TBinaryProtocol(transport));
// Server:
TServerTransport transport = new TServerSocket(8888);
TServer server = new TSimpleServer(new Args(transport).processor(new DecoratedProcessor(...)));
But I'm not sure what to do inside DecoratedProtocol and DecoratedProcessor?
There is a solution to add message meta data called the THeaderProtocol, but AFAIK right now this is only implemented for C++. That could be a starting point to implement the same for Java.
EDIT: I just noticed that fbthrift seems to implement THeaderProtocol for Java already.
I'm executing Apache Thrift tutorial for Java.
When running 2 client processes at the same time, the server doesn't accept the 2nd client. Only after the first client finishes, the second one is accepted by the server.
Can anyone explain what's going on?
How can I make the server accept several connections in several threads?
Can anyone explain what's going on?
You already found it out: The TSimpleServer allows only for one connection at a time. It will be available again when the first client disconnects.
How can I make the server accept several connections in several threads?
Use one of the threading servers, whichever fits your use case best.
TThreadPoolServer
TThreadedSelectorServer
TNonBlockingServer
The half-sync/half-async server
Please note, that some of the servers require the client to use TFramedTransport.
Based on other answers, below is the code to enable executing multiple clients simultaneously.
Server (simple):
CalculatorHandler handler = new CalculatorHandler();
Calculator.Processor processor = new Calculator.Processor(handler);
TNonblockingServerSocket serverTransport = new TNonblockingServerSocket(9090);
THsHaServer.Args args = new THsHaServer.Args(serverTransport);
args.processor(processor);
args.transportFactory(new TFramedTransport.Factory());
TServer server = new THsHaServer(args);
server.serve();
Client:
transport = new TSocket("localhost", 9090);
transport.open();
TProtocol protocol = new TBinaryProtocol(new TFramedTransport(transport));
Calculator.Client client = new Calculator.Client(protocol);
perform(client);
I have a Jersey client up and running, using the Apache Client 4 library, like this:
private Client createClient() {
ApacheHttpClient4Config cc = new DefaultApacheHttpClient4Config();
// boring stuff here
return ApacheHttpClient4.create(cc);
}
But this by default uses a BasicClientConnManager, which doesn't allow multi-threaded connections.
The ApacheHttpClient4Config Javadoc says that I need to set the PROPERTY_CONNECTION_MANAGER to a ThreadSafeClientConnManager instance if I want multi-threaded operation. I can do this, and it works OK:
private Client createClient() {
ApacheHttpClient4Config cc = new DefaultApacheHttpClient4Config();
cc.getProperties().put(ApacheHttpClient4Config.PROPERTY_CONNECTION_MANAGER,
new ThreadSafeClientConnManager());
// boring stuff here
return ApacheHttpClient4.create(cc);
}
But ThreadSafeClientConnManager is deprecated. This is annoying.
The more modern version is PoolingHttpClientConnectionManager. Unfortunately, though, the ApacheHttpClient4.create() method requires the connection manager to be an implementation of ClientConnectionManager (itself deprecated), and PoolingHttpClientConnectionManager doesn't implement that interface. So if I try to use it, my connection manager gets ignored and we're back to a BasicClientConnManager.
How can I end up with a thread-safe client without using anything that's deprecated?
You can create the client as follows (see https://github.com/phillbarber/connection-leak-test/blob/master/src/test/java/com/github/phillbarber/connectionleak/IntegrationTestThatExaminesConnectionPoolBeforeAndAfterRun.java#L30-L33):
client = new ApacheHttpClient4(new ApacheHttpClient4Handler(HttpClients.custom()
.setConnectionManager(new PoolingHttpClientConnectionManager())
.build(), null, false));
Following Jetty documentation and answer to this question creation of websocket client is as simple as
WebSocketClient client = new WebSocketClient();
SimpleEchoSocket socket = new SimpleEchoSocket();
try {
client.start();
URI echoUri = new URI(destUri);
ClientUpgradeRequest request = new ClientUpgradeRequest();
client.connect(socket, echoUri, request);
System.out.printf("Connecting to : %s%n", echoUri);
socket.awaitClose(5, TimeUnit.SECONDS);
}
But I can't find SimpleEchoSocket! I try several versions of org.eclipse.jetty.websocket:websocket-client but had no success. Looks like documentation is outdated, but maybe I am doing something wrong? How can I use this example from Jetty doc?
SimpleEchoSocket is your socket implementation.
In other words, its your code, your class.
Something that implements the on open, on close, on message logic in the ways that the same documentation explains.
You have a few choices here, it can be jetty api specific, or jsr-356 (aka javax.websocket) specific.
Then you choose between a traditional class that implements an interface, or one that is marked up with annotations.
I am using Apache Mina in the Server side. I've a client which is written in tradition IO. Here's the CLIENT side code that sends data to server.
class SomeClass extends Thread
{
Socket socket;
//Constructor
SomeClass()
{
Socket socket = ...
}
public void run()
{
while (j++ < 10)
{
System.out.println("CLIENT[" + clientNo + "] Send Message =>" + requests[clientNo][j]);
OutputStream oStrm = socket.getOutputStream();
byte[] byteSendBuffer = (requests[clientNo][j]).getBytes();
oStrm.write(byteSendBuffer);
oStrm.flush();
}
}
}
The above thread is run for say 20 times. So 20 sockets are created. And in 1 socket, many messages are send. With a server written using IO socket classes i'm able to retrieve data perfectly.
THe problem comes in the Apache Mina based Server which uses BUFFER! I am not able to get individual messages.
How do i get individual messages (given i'm not able to change anything in client, AND the length of individual messages are not known)
Server Side Code
Socket Creation
public static void main(String[] args) throws IOException, SQLException {
System.out.println(Charset.defaultCharset().name());
IoAcceptor acceptor = new NioSocketAcceptor();
ProtocolCodecFilter(charset.newEncoder(),charset.newDecoder() ));
acceptor.setHandler( new TimeServerHandler() );
acceptor.getSessionConfig().setReadBufferSize(64 );
acceptor.getSessionConfig().setIdleTime( IdleStatus.BOTH_IDLE, 10 );
acceptor.bind( new InetSocketAddress(PORT) );
}
Handler Code
public void messageReceived(IoSession session, Object message) throws Exception {
AbstractIoBuffer bf = (AbstractIoBuffer)message;
Charset charset = Charset.forName("UTF-8");
CharsetDecoder decoder = charset.newDecoder();
String outString = bf.getString(decoder);
}
How do i get individual messages
You don't. There is no such thing as a message in TCP. It is a byte-stream protocol. There are no message boundaries and there is no guarantee that one read equals one write at the other end.
(given i'm not able to change anything in client, AND the length of individual messages are not known)
Your are going to have to parse the messages to find where they stop according to the definition of the application protocol. If that isn't possible because, say, the protocol is ambiguous, the client will have to be junked. However it seems that as you can't change the client, it must already work with an existing system, so the guy before you had the same problem and solved it somehow.
MINA is actually a very elaborate framework to solve your problem in an elegant way. Its basic concept is a filter chain, in which a series of filters are applied on an incoming message.
You should implement a protocol decoder (implementing MessageDecoder) and register it in your MINA filter chain. That decoder should parse byte buffers to the object representation of your choice.
Then, you can register a message handler that handles complete messages.