Example on how to use TFileTransport in Thrift (Client/Server) - java

Is there anyone who managed to get TFileTransport as a transport layer, to work? I've tried but since there is no documentation (or have I not found it?) for this, I am not able to make it work.
If anyone have been more successful and could provide some sample code, it would be great.
edit:
What I've tried so far:
public class FileThriftServer {
public static void startThriftServer(
ThriftDataBenchmark.Processor<ThriftDataBenchmarkHandler> processor) {
try {
File input = new File("ThriftFile.in");
if(!input.exists()){
input.createNewFile();
}
File output = new File("ThriftFile.out");
if(!output.exists()){
output.createNewFile();
}
TFileTransport inputFileTransport = new TFileTransport(input.getAbsolutePath(), true);
TFileTransport outputFileTransport = new TFileTransport(output.getAbsolutePath(), false);
inputFileTransport.open();
outputFileTransport.open();
TFileProcessor fProcessor =
new TFileProcessor(processor, new TJSONProtocol.Factory(), inputFileTransport, outputFileTransport);
// this results in error in case I don't call those open methods above
fProcessor.processChunk();
System.out.println("File Thrift service started ...");
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
// ThriftDataBenchmarkHandler is an implementation of my test service
startThriftServer(new ThriftDataBenchmark.Processor<ThriftDataBenchmarkHandler>(
new ThriftDataBenchmarkHandler()));
}
}
Now I don't know if I am even on a good way, maybe I misunderstood the concept of this transport (again, it is not documented). I would expect I start the server by some method now which will listen on the input file. When clients put there something, it would process it and write the answer to output file (I didn't try to write client yet since this peace of code just executes and exists, it is obviously not right).
edit 2:
Ok, so if I understand it right, this code is ok and it should process one request of the client, if it's there. So I am moving to the client side, doing something like this:
File input = new File(THRIFT_INPUT_FILE_PATH);
if (!input.exists()) {
input.createNewFile();
}
TTransport transport = new TFileTransport(input.getAbsolutePath(),
false);
TProtocol protocol = new TJSONProtocol(transport);
ThriftDataBenchmark.Client client = new ThriftDataBenchmark.Client(
protocol);
// my testing service, the parameters are not important
SimpleCompany company = client.getSimpleCompanyData("token", 42);
Unfortunatelly calling getSimpleCompanyData results in:
org.apache.thrift.transport.TTransportException: Not Supported
at org.apache.thrift.transport.TFileTransport.write(TFileTransport.java:572)
at org.apache.thrift.transport.TTransport.write(TTransport.java:105)
at org.apache.thrift.protocol.TJSONProtocol.writeJSONArrayStart(TJSONProtocol.java:476)
at org.apache.thrift.protocol.TJSONProtocol.writeMessageBegin(TJSONProtocol.java:487)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
It's a bit confusing that server side requires input and output transport but on the client side, it only accepts one. How does it read an answer and from where?
Let's not move into some extra logic of checking the file for changes, if it's not already part of Thrift. I'll be ok at this point by doing it manually in sense of: running the client first, then running the server side.

I would expect I start the server by some method now which will listen on the input file. When clients put there something, it would process it and write the answer to output file (I didn't try to write client yet since this peace of code just executes and exists, it is obviously not right).
That's exactly right. In particular, the fProcessor.processChunk() call you used will process exactly one chunk (the current one). The whole class looks as designed around the assumption that the file size is static and does not change over time. However, the underlying TFileTransport supports what's called a tailPolicy, used when a read call hits EOF:
public class TFileTransport extends TTransport {
public static enum tailPolicy {
NOWAIT(0, 0),
WAIT_FOREVER(500, -1);
/**
* Time in milliseconds to sleep before next read
* If 0, no sleep
*/
public final int timeout_;
/**
* Number of retries before giving up
* if 0, no retries
* if -1, retry forever
*/
public final int retries_;
// ... ctor ...
}
/**
* Current tailing policy
*/
tailPolicy currentPolicy_ = tailPolicy.NOWAIT;
Another option to get it to work could be calling fProcessor.processChunk(int chunkNum), watching the file contents separately and repeat the calls when new data come in. It's certainly not such a bad idea to use the TFileProcessor as a starting point and improve it as needed.
// this results in error in case I don't call those open methods above
fProcessor.processChunk();
Opening the transports before using is fine. I think that part is ok.
org.apache.thrift.transport.TTransportException: Not Supported
at org.apache.thrift.transport.TFileTransport.write(TFileTransport.java:572)
at org.apache.thrift.transport.TTransport.write(TTransport.java:105)
Unfortunately, that seems pretty correct yet. The only place where writing is implemented is the code in the C++ library. Both Java and D only support reading (yet).

Related

Use code generation for executing generic tests

I think I have an interesting question and PERHAPS there is already the answer which is still a secret for me, so I hope to get some helps from expers. :)
So here is the thing:
I work for the test/validation team to test our Java API and basically my job is to follow test plan and write the test code. After writing that for more than two months, I find the codes are really similar. For example:
To test function could return expected result or throw exception correctly, we may need write several .java to run.
1.java set up server connection, connect client and send request, initiate variables with correct values and pass them to the function A, catch the answer and analyse it
2.java set up server connection, connect client and send request, initiate all variables with correct values but one with bad value and pass them to function A, catch the answer and analyse it
3.java set up server connection, connect client and send request, initiate all variables with correct values but two with bad values and pass them to function A, catch the answer and analyse it
so you see in three java test files, the most part of them are the same or similar enough and even copy/paste make the job boring and possible to be wrong.
I wonder whether or not I could define test code corresponding different behavior, then for every test java file, I define a text including the behavior and then a mother class who is in charge of loading the text file and assembling the final test java file according to the text file?
Like this:
Text File:
1) set up server
2) connect client
3) send request
4) initiate variables with correct values
5) initiate variables with correct values but one with bad value
6) initiate variables with correct values but two with bad values
7) catch the result and analyse it
Mother.java
1) load Text file
2) create a son.java
3) find the code corresponding the Text file and write them to son.java
Then the coder open son.java at IDE to check syntax, or import or anything conflict then run it.
Is my idea realizable or not? Is there already something similar?
Any information would be appreciated, thanks a millions in advance!
Honestly, this does not sound like a good use case for code generation. Instead of generating a class for each test case, you should implement a more general testing utility which takes the required input as its data and executes the generic testing code based on this data.
From what you write, this would for example be something like a simple base class for a JUnit test:
abstract class AbstractServerDependantTest {
protected Server server;
protected Client client;
#Before
public void setUp() {
server = new Server();
server.start();
client = new Client();
client.connectTo(server);
}
#After
public void tearDpwm() {
client.disconnect();
server.shutDown();
}
}
Now you can write three test classes which inherit from this AbstractServerDependantTest without copy pasting your code.

Networked GUI handler for binary protocol

Alright, so I created a Client and Server that manipulates a Map via a GUI interface using a text based protocol. The handler below was used to create a "work order" to manipulate the GUI on a separate thread from the network communications.
class RemoteInputHandler implements Runnable,SharedVariables
{
#Override
public void run()
{
try
{
String input = netComm.reader.readLine();
while (input != null)
{
// Make a separate copy of the input string
String inputCopy = input;
// Post a work order to process the command on the GUI thread
Platform.runLater(() ->
{
handleRemote(inputCopy);
});
// Get the next remote input
input = netComm.reader.readLine();
}
} catch (IOException ex)
{
throw new RuntimeException(ex);
}
}
This pulls the next line of input from the server without freezing up the GUI. I then use the input in the handleRemote() method with a scanner to determine what is done with the input. The string retrieved from the reader looked something like "put key value". I'd then get the first word "put", using a scanner, to get the "command" and use a switch statement to determine how the GUI / Map should be updated on the client and server side respectively.
I'm doing another GUI based program that uses a binary protocol instead, but I'm having trouble figuring out how to handle the information in the same way as I did with the readLine() above. Is there a way to do this? Am i thinking about it the wrong way? I thought I could just get all of the bytes into an array, but I'm having trouble even figuring that out.
I could really use a hint! Thank you!

Messaging between Java and Flex with Flerry

I'm currently working on a project that need to communicate with java from air without the use of a server like Tomcat. For this i found and use Flerry.
Communicating between Java and Air is no problem, as long as I try to send a message from the class that I initially instantiated from Air.
What I try to do is subscript to messages from a Message Class in java and use that class to send info and errors to Air from Java.
MessageController.java:
public class MessageController
{
public MessageController()
{
}
public static void sendErrorMessage(String errorMessage)
{
NativeObject.sendMessage(errorMessage, "error");
}
public static void sendInfoMessage(String infoMessage)
{
NativeObject.sendMessage(infoMessage, "info");
}
}
In Air I create a NativeObject on the messaging class and subscribe to the messages:
var messageController:NativeObject = new NativeObject();
messageController.source = "controller.MessageController" ;
messageController.singleton = true;
messageController.debug = false;
messageController.addEventListener(FaultEvent.FAULT, onFileControllerFault, false, 0, true);
messageController.subscribe("info", infoMessageHandler);
messageController.subscribe("error", errorMessageHandler);
As you would have guessed, this doesn't work.
It seems that I am only able to dispatch messages from the class that I subscribe to directly, for example if I do this:
messageController.start();
and in my MessageController.java i put this:
public void start()
{
NativeObject.sendMessage("test message", "info");
}
the infoMessageHandler receives an message containing test message, as it should.
How can I dispatch messages from whatever class in Java and catch them on the Air side?
I'm not sure I fully understand the issue yet, but is there a reason you can't simply send and receive all messages through a Java 'communication' class? If that's working, I'd just set up public methods in Java interfacing/communications class and be done with it.
(It's been awhile since I've dug into how Flerry works so I can't recall enough to point out the likely cause of the behavior you're seeing.) Also Flerry is open source, and not very big at all. If you really want to know why it's behaving in a certain way, I'd wager you could figure it out by looking at the source. (Won't take 6 months to learn like some Spring or Hibernate code base, or even BlazeDS, I promise.)
I do use Flerry in a small app, but now that I think about it, I only have one class which dispatches anything to Flex! But I feel like you may need to describe the problem you're facing differently, because it sounds like expected behavior to me.

Java Multithreaded I/0 and communication problem

I am using java to create an application for network management. In this application I establish communication with network devices using SNMP4j library (for the snmp protocol). So, Im supposed to scan certain values of the network devices using this protocol and put the result into a file for caching. Up in some point I decided to make my application multi-threaded and assign a device to a thread. I created a class that implements the runnable interface and then scans for the values that I want for each device.
When i run this class alone it, works fine. but when I put multiple threads at the same time the output mess up, it prints additional or out of order output into the files. Now, i wonder if this problem is due to the I/O or due to the communication.
Here I'll put some of the code so that you can see what im doing and help me figure what's wrong.
public class DeviceScanner implements Runnable{
private final SNMPCommunicator comm;
private OutputStreamWriter out;
public DeviceScanner(String ip, OutputStream output) throws IOException {
this.device=ip;
this.comm = new SNMPV1Communicator(device);
oids=MIB2.ifTableHeaders;
out = new OutputStreamWriter(output);
}
#Override
public void run(){
//Here I use the communicator to request for desired data goes something like ...
String read=""
for (int j=0; j<num; j++){
read= comm.snmpGetNext(oids);
out.write(read);
this.updateHeaders(read);
}
out.flush();
//...
}
}
some of the expected ooutput would be something like:
1.3.6.1.2.1.1.1.0 = SmartSTACK ELS100-S24TX2M
1.3.6.1.2.1.1.2.0 = 1.3.6.1.4.1.52.3.9.1.10.7
1.3.6.1.2.1.1.3.0 = 26 days, 22:35:02.31
1.3.6.1.2.1.1.4.0 = admin
1.3.6.1.2.1.1.5.0 = els
1.3.6.1.2.1.1.6.0 = Computer Room
but instead i get something like (varies):
1.3.6.1.2.1.1.1.0 = SmartSTACK ELS100-S24TX2M
1.3.6.1.2.1.1.2.0 = 1.3.6.1.4.1.52.3.9.1.10.7
1.3.6.1.2.1.1.4.0 = admin
1.3.6.1.2.1.1.5.0 = els
1.3.6.1.2.1.1.3.0 = 26 days, 22:35:02.31
1.3.6.1.2.1.1.6.0 = Computer Room
1.3.6.1.2.1.1.1.0 = SmartSTACK ELS100-S24TX2M
1.3.6.1.2.1.1.2.0 = 1.3.6.1.4.1.52.3.9.1.10.7
*Currently I have one file per device scanner desired.
i get them from a list of ip , it looks like this. Im also using a little threadpool to keep a limited number of threads at the same time .
for (String s: ips){
output= new FileOutputStream(new File(path+s));
threadpool.add(new DeviceScanner(s, output));
}
I suspect SNMPV1Communicator(device) is not thread-safe. As I can see it's not a part of SNMP4j library.
Taking a wild guess at what's going on here, try putting everything inside a synchronized() block, like this:
synchronized (DeviceScanner.class)
{
for (int j=0; j<num; j++){
read= comm.snmpGetNext(oids);
out.write(read);
this.updateHeaders(read);
}
out.flush();
}
If this works, my guess is right and the reason for the problems you're seeing is that you have many OutputStreamWriters (one on each thread), all writing to a single OutputStream. Each OutputStreamWriter has its own buffer. When this buffer is full, it passes the data to the OutputStream. It's essentially random when each each OutputStreamWriter's buffer is full - it might well be in the middle of a line.
The synchronized block above means that only one thread at a time can be writing to that thread's OutputStreamWriter. The flush() at the end means that before leaving the synchronized block, the OutputStreamWriter's buffer should have been flushed to the underlying OutputStream.
Note that synchronizing in this way on the class object isn't what I'd consider best practice. You should probably be looking at using a single instance of some other kind of stream class - or something like a LinkedBlockingQueue, with all of the SNMP threads passing their data over to a single file-writing thread. I've added the synchronized as above because it was the only thing available to synchronize on within your pasted example code.
You've got multiple threads all using buffered output, and to the same file.
There's no guarantees as to when those threads will be scheduled to run ... the output will be fairly random ordered, dictated by the thread scheduling.

How to allow running only one instance of a Java program at a time?

I need to prevent users from starting my Java application (WebStart Swing app) multiple times. So if the application is already running it shouldn't be possible to start it again or show a warning / be closed again.
Is there some convenient way to achieve this? I thought about blocking a port or write sth to a file. But hopefully you can access some system properties or the JVM?
btw. target platform is Windows XP with Java 1.5
I think your suggestion of opening a port to listen when you start your application is the best idea.
It's very easy to do and you don't need to worry about cleaning it up when you close your application. For example, if you write to a file but someone then kills the processes using Task Manager the file won't get deleted.
Also, if I remember correctly there is no easy way of getting the PID of a Java process from inside the JVM so don't try and formulate a solution using PIDs.
Something like this should do the trick:
private static final int PORT = 9999;
private static ServerSocket socket;
private static void checkIfRunning() {
try {
//Bind to localhost adapter with a zero connection queue
socket = new ServerSocket(PORT,0,InetAddress.getByAddress(new byte[] {127,0,0,1}));
}
catch (BindException e) {
System.err.println("Already running.");
System.exit(1);
}
catch (IOException e) {
System.err.println("Unexpected error.");
e.printStackTrace();
System.exit(2);
}
}
This sample code explicitly binds to 127.0.0.1 which should avoid any firewall warnings, as any traffic on this address must be from the local system.
When picking a port try to avoid one mentioned in the list of Well Known Ports. You should ideally make the port used configurable in a file or via a command line switch in case of conflicts.
As the question states that WebStart is being used, the obvious solution is to use javax.jnlp.SingleInstanceService.
This service is available in 1.5. Note that 1.5 is currently most of the way through its End Of Service Life period. Get with Java SE 6!
I think that the better idea would be to use file lock (quite an old idea :) ). Since Java 1.4 a new I/O library was introduced, that allows file locking.
Once the application starts it tries to acquire lock on a file (or create it if does not exist), when the application exits the lock is relased. If application cannot acquire a lock, it quits.
The example how to do file locking is for example in Java Developers Almanac.
If you want to use file locking in Java Web Start application or an applet you need to sing the application or the applet.
You can use JUnique library. It provides support for running single-instance java application and is open-source.
http://www.sauronsoftware.it/projects/junique/
See also my full answer at How to implement a single instance Java application?
We do the same in C++ by creating a kernal mutex object and looking for it at start up. The advantages are the same as using a socket, ie when the process dies/crashes/exits/is killed, the mutex object is cleaned up by the kernel.
I'm not a Java programmer, so I am not sure whether you can do the same kind of thing in Java?
I've create the cross platform AppLock class.
http://mixeddev.info/articles/2015/02/01/run-single-jvm-app-instance.html
It is using file lock technique.
Update. At 2016-10-14 I've created package compatible with maven/gradle https://github.com/jneat/jneat and explained it here http://mixeddev.info/articles/2015/06/01/synchronize-different-jvm-instances.html
You could use the registry, although this halfheartedly defeats the purpose of using a high-level language like java. At least your target platform is windows =D
Try JUnique:
String appId = "com.example.win.run.main";
boolean alreadyRunning;
try {
JUnique.acquireLock(appId);
alreadyRunning = false;
} catch (AlreadyLockedException e) {
alreadyRunning = true;
}
if (alreadyRunning) {
Sysout("An Instance of this app is already running");
System.exit(1);
}
I've seen so many of this questions and I was looking to solve the same problem in a platform independent way that doesn't take the chance to collide with firewalls or get into socket stuff.
So, here's what I did:
import java.io.File;
import java.io.IOException;
/**
* This static class is in charge of file-locking the program
* so no more than one instance can be run at the same time.
* #author nirei
*/
public class SingleInstanceLock {
private static final String LOCK_FILEPATH = System.getProperty("java.io.tmpdir") + File.separator + "lector.lock";
private static final File lock = new File(LOCK_FILEPATH);
private static boolean locked = false;
private SingleInstanceLock() {}
/**
* Creates the lock file if it's not present and requests its deletion on
* program termination or informs that the program is already running if
* that's the case.
* #return true - if the operation was succesful or if the program already has the lock.<br>
* false - if the program is already running
* #throws IOException if the lock file cannot be created.
*/
public static boolean lock() throws IOException {
if(locked) return true;
if(lock.exists()) return false;
lock.createNewFile();
lock.deleteOnExit();
locked = true;
return true;
}
}
Using System.getProperty("java.io.tmpdir") for the lockfile path makes sure that you will always create your lock on the same place.
Then, from your program you just call something like:
blah blah main(blah blah blah) {
try() {
if(!SingleInstanceLock.lock()) {
System.out.println("The program is already running");
System.exit(0);
}
} catch (IOException e) {
System.err.println("Couldn't create lock file or w/e");
System.exit(1);
}
}
And that does it for me. Now, if you kill the program it won't delete the lock file but you can solve this by writing the program's PID into the lockfile and making the lock() method check if that process is already running. This is left as an assingment for anyone interested. :)

Categories

Resources