Develop Reference

Can I add additional field (metadata) to thrift message?

I have generated client and server.
I need to add one more field to message in client and read it in server.
I think about to decorate TProtocol on client side and TProcessor on server side. E.g.:
// Client:
TTransport transport = new TSocket("localhost", 8888);
transport.open();
TProtocol protocol = new DecoratedProtocol(new TBinaryProtocol(transport));
// Server:
TServerTransport transport = new TServerSocket(8888);
TServer server = new TSimpleServer(new Args(transport).processor(new DecoratedProcessor(...)));
But I'm not sure what to do inside DecoratedProtocol and DecoratedProcessor?

There is a solution to add message meta data called the THeaderProtocol, but AFAIK right now this is only implemented for C++. That could be a starting point to implement the same for Java.
EDIT: I just noticed that fbthrift seems to implement THeaderProtocol for Java already.

ActiveMQ Java Broker, Python Client

I have for legacy reasons a Java activeMQ implementation of the Broker/Publisher over vanilla tcp transport protocol. I wish to connect a Python client to it, however all the "stomp" based documentation doesn't seem to have it, not over the stomp protcol, and when I try the basic examples I get the error on the Java Broker side:
[ActiveMQ Transport: tcp:///127.0.0.1:62860#5001] WARN org.apache.activemq.broker.TransportConnection.Transport - Transport Connection to: tcp://127.0.0.1:62860 failed: java.io.IOException: Unknown data type: 80
The Broker code is very vanilla in Java:
String localVMurl = "vm://localhost";
String remoterURL = "tcp://localhost:5001";
BrokerService broker = new BrokerService();
broker.addConnector(localVMurl);
broker.addConnector(remoterURL);
broker.setAdvisorySupport(true);
broker.start();
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(localVMurl+"?create=false");
Connection connection = connectionFactory.createConnection();
and the Python just fails. I can't seem to find anything online using just basic "tcp://localhost:" connections from Python. Am I doing something wrong here?
import stomp
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
conn = stomp.Connection(host_and_ports = [('localhost', 5001)])
conn.start()
conn.connect('admin', 'password', wait=True)
and I get the error:
IndexError: list index out of range

Without seeing the broker configuration it is a bit tricky to answer but from the error I'd guess you are trying to connect a STOMP client to the OpenWire transport which won't work, you need to have a STOMP TransportConnector configured on the broker and point the STOMP client there.
See the ActiveMQ STOMP documentation.
To add STOMP support for an embedded broker you'd do something along the lines of:
brokerService.addConnector("stomp://0.0.0.0:61613");

Asynchronous Thrift Java Client over UNIX Domain Socket

I have an application consisting of a Java Thrift Asynchronous client communicating with a Python/Twisted Thrift Asynchronous server over TCP using the loopback interface (localhost).
I want to use a UNIX domain socket instead of a TCP socket for increased throughput, but I cannot find a suitable Asynchronous (Non-blocking) UNIX Domain Socket implementation for Java to use with Thrift.
I had this Python/Twisted TCP Thrift server:
handler = ThriftServiceHandler(am)
processor = ThriftServiceHandler.Processor(handler)
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
reactor.listenTCP(PORT, TTwisted.ThriftServerFactory(processor, pfactory), interface="127.0.0.1")
And I managed to create an Asynchronous UNIX Domain Socket Thrift server with:
handler = ThriftServiceHandler(am)
processor = ThriftServiceHandler.Processor(handler)
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
reactor.listenUNIX("thrift.sock", TTwisted.ThriftServerFactory(processor, pfactory))
Currently I have this Java client:
clientManager = new TAsyncClientManager();
factory = new TBinaryProtocol.Factory();
socket = new TNonblockingSocket("127.0.0.1", thriftConstants.PORT);
client = new ThriftService.AsyncClient(this.factory, this.clientManager, this.socket);
But I am missing how to do the Java Client implementation. I looked at using https://github.com/jnr/jnr-unixsocket and https://github.com/kohlschutter/junixsocket, but I couldn't get them to work with Thrift's Java AsyncClient. Any help would be appreciated.
I am using the latest version of Thrift (0.9.2).

TTransportException when using TFramedTransport

I'm pretty puzzled with this issue. I have an Apache Thrift 0.9.0 client and server. The client code goes like this:
this.transport = new TSocket(this.server, this.port);
final TProtocol protocol = new TBinaryProtocol(this.transport);
this.client = new ZKProtoService.Client(protocol);
This works fine. However, if I try to wrap the transport in a TFramedTransport
this.transport = new TSocket(this.server, this.port);
final TProtocol protocol = new TBinaryProtocol(new TFramedTransport(this.transport));
this.client = new ZKProtoService.Client(protocol);
I get the following obscure (no explanation message whatsoever) exception in the client side. Server side shows no error.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at com.blablabla.android.core.device.proto.ProtoService$Client.recv_open(ProtoService.java:108)
at com.blablabla.android.core.device.proto.ProtoService$Client.open(ProtoService.java:95)
at com.blablabla.simpleprotoclient.proto.ProtoClient.initializeCommunication(ProtoClient.java:411)
at com.blablabla.simpleprotoclient.proto.ProtoClient.doWork(ProtoClient.java:269)
at com.blablabla.simpleprotoclient.proto.ProtoClient.run(ProtoClient.java:499)
at java.lang.Thread.run(Thread.java:724)
It also fails if I use TCompactProtocol instead of TBinaryProtocol.
In the server side I have extended TProcessor with my own class since I need to reuse existing service handler (the service server-side IFace implementation) for this client:
#Override
public boolean process(final TProtocol in, final TProtocol out)
throws TException {
final TTransport t = in.getTransport();
final TSocket socket = (TSocket) t;
socket.setTimeout(ProtoServer.SOCKET_TIMEOUT);
final String clientAddress = socket.getSocket().getInetAddress()
.getHostAddress();
final int clientPort = socket.getSocket().getPort();
final String clientRemote = clientAddress + ":" + clientPort;
ProtoService.Processor<ProtoServiceHandler> processor = PROCESSORS
.get(clientRemote);
if (processor == null) {
final ProtoServiceHandler handler = new ProtoServiceHandler(
clientRemote);
processor = new ProtoService.Processor<ProtoServiceHandler>(
handler);
PROCESSORS.put(clientRemote, processor);
HANDLERS.put(clientRemote, handler);
ProtoClientConnectionChecker.addNewConnection(clientRemote,
socket);
}
return processor.process(in, out);
}
And this is how I start the server side:
TServerTransport serverTransport = new TServerSocket(DEFAULT_CONTROL_PORT);
TServer server = new TThreadPoolServer(new TThreadPoolServer.Args(
serverTransport).processor(new ControlProcessor()));
Thread thControlServer = new Thread(new StartServer("Control", server));
thControlServer.start();
I have some questions:
Is it correct to reuse service handler instances or I shouldn't be doing this?
Why does it fail when I use TFramedTransport or TCompactProtocol? How to fix this?
Any help on this issue is welcome. Thanks in advance!

I was having the same problem and finally found the answer. It is possible to set the transport type on the server, though this is not clear from most tutorials and examples I've found on the web. Have a look at all of the methods of the TServer.Args class (or the args classes for other servers, which extend TServer.Args). There are methods inputTransportFactory and outputTransportFactory. You can use new TFramedTransport.Factory() as inputs to each of these methods to declare which transport the server should use. In scala:
val handler = new ServiceStatusHandler
val processor = new ServiceStatus.Processor(handler)
val serverTransport = new TServerSocket(9090)
val args = new TServer.Args(serverTransport)
.processor(processor)
.inputTransportFactory(new TFramedTransport.Factory)
.outputTransportFactory(new TFramedTransport.Factory)
val server = new TSimpleServer(args)
println("Starting the simple server...")
server.serve()
Note that if you are using a TAsyncClient, you have no choice about the transport that you use. You must use TNonblockingTransport, which has only one standard implementation, TNonblockingSocket, which internally wraps whatever protocol you are using in a framed transport. It doesn't actually wrap your chosen protocol in a TFramedTransport, but it does prepend the length of the frame to the content that it writes, and expects the server to prepend the length of the response as well. This wasn't documented anywhere I found, but if you look at the source code and experiment with different combinations, you will find that with TSimpleServer you must use TFramedTransport to get it to work with an async client.
By the way, it's also worth noting that the docs say that a TNonblockingServer must use TFramedTransport in the outermost later of the transport. However, the examples don't show this being set in TNonblockingServer.Args, yet you still find that you must use TFramedTransport on the client side to successfully execute an rpc on the server. This is because TNonblockingServer.Args has its input and output protocols set to TFramedTransport by default (you can see this using reflection to inspect the fields of the superclass hierarchy or in the source code for the constructor of AbstractNonblockingServerArgs -- you can override the input and output transports, but the server will likely fail for the reasons discussed in the documentation).

When the issue happens with framed, but it works without framed, then you have an incompatible protocol stack on both ends. Choose one of the following:
either modify the server code to use framed as well
or do not use framed on the client
A good rule of thumb is, to always use the exact same protocol/transport stack on both ends. In the particular case it blows up, because framed adds a four-byte header holding the size of the message that follows. If the server does not use framed, these additional four bytes sent by the client will be interpreted (wrongly) as part of the message.
Altough the sample code in that answer
TNonblockingServer in thrift crashes when TFramedTransport opens is for C++, adding framed on the server should be very similar with Java.
PS: Yes, it is perfectly ok to re-use your handler. A typical handler is a stateless thing.

Making a PHP server to read a java based client

public class Client {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws IOException {
Socket sock = new Socket();
sock.connect(new InetSocketAddress(InetAddress.getLocalHost(), 43594));
DataOutputStream out = new DataOutputStream(sock.getOutputStream());
DataInputStream in = new DataInputStream(sock.getInputStream());
out.writeByte(255);
out.writeLong(133713371337L);
System.out.println("Response: " + in.readByte() + ", " + in.readLong());
sock.close();
}
and the PHP server meant to read and write back to the java client:
<?php
require 'Stream.php';
$stream = new Stream();
$address = '127.0.0.1';
$port = 43594;
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
socket_bind($sock, 0, $port) or die('Could not bind to address');
socket_listen($sock);
echo "Server listening on port " . $port . "...";
$client = socket_accept($sock);
$data = socket_read($client, 1, PHP_BINARY_READ);
$byte_array = unpack('C*', $data);
$stream->setStream($byte_array);
$b2 = $stream->readUnsignedByte();
$l1 = socket_read($client, $stream->readQWord());
echo $b1 . " " . $l1;
socket_write($client, 0);
socket_write($client, $stream->writeQWord(2148000000));
The stream class being used: http://pastebin.com/p1vc9aPG
I'm trying to write a client in java that contacts, sends and reads data to/from the server. The server is written in PHP and ran via cli, while the client is written in java. Please help if you can, there probably aren't many people who can help with this issue.
The protocol is already established and I'm just trying to implement it in PHP. I can't seem to read/write back to the java client from the PHP page.
The client side in Java is defined how I want it to be. What I'm looking for is the equivalent server in PHP and I can't seem to get it to work.

I've actually written this the other way around, the client was a PHP web app and the server was a java daemon under jsvc.
The first consideration should be defining a transfer protocol, since the types and object representations of the two languages are different and they might be running on hardware with different numeric representations. I'd recommend JSON or XML.
After that your server probably needs to be able to handle multiple connections, so threading or non-blocking socket I/O will be needed.

First, you should call .flush() after you write to the socket in your Java code. The data won't get sent to the server otherwise.
Then, communications between network endpoints is just about agreement on the protocol used. There is nothing language specific here...