Develop Reference

RxNetty reuse the connection

I want to use Netflix-Ribbon as TCP client load balancer without Spring Cloud,and i write test code.
public class App implements Runnable
{
public static String msg = "hello world";
public BaseLoadBalancer lb;
public RxClient<ByteBuf, ByteBuf > client;
public Server echo;
App(){
lb = new BaseLoadBalancer();
echo = new Server("localhost", 8000);
lb.setServersList(Lists.newArrayList(echo));
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
client = RibbonTransport.newTcpClient(lb, impl);
}
public static void main( String[] args ) throws Exception
{
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(new App());
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
public String sendAndRecvByRibbon(final String data)
{
String response = "";
try {
response = client.connect().flatMap(new Func1<ObservableConnection<ByteBuf, ByteBuf>,
Observable<ByteBuf>>() {
public Observable<ByteBuf> call(ObservableConnection<ByteBuf, ByteBuf> connection) {
connection.writeStringAndFlush(data);
return connection.getInput();
}
}).timeout(1, TimeUnit.SECONDS).retry(1).take(1)
.map(new Func1<ByteBuf, String>() {
public String call(ByteBuf ByteBuf) {
return ByteBuf.toString(Charset.defaultCharset());
}
})
.toBlocking()
.first();
}
catch (Exception e) {
System.out.println(((LoadBalancingRxClientWithPoolOptions) client).getMaxConcurrentRequests());
System.out.println(lb.getLoadBalancerStats());
}
return response;
}
public void run() {
for (int i = 0; i < 200; i++) {
sendAndRecvByRibbon(msg);
}
}
}
i find it will create a new socket everytime i callsendAndRecvByRibbon even though the poolEnabled is setting to true. So,it confuse me,i miss something?
and there are no option to configure the size of the pool,but hava a PoolMaxThreads and MaxConnectionsPerHost.
My question is how to use a connection pool in my simple code, and what's wrong with my sendAndRecvByRibbon,it open a socket then use it only once,how can i reuse the connection?thanks for your time.
the server is just a simple echo server writing in pyhton3,i comment outconn.close() because i want to use long connection.
import socket
import threading
import time
import socketserver
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
conn = self.request
while True:
client_data = conn.recv(1024)
if not client_data:
time.sleep(5)
conn.sendall(client_data)
# conn.close()
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
if __name__ == "__main__":
HOST, PORT = "localhost", 8000
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
server.serve_forever()
and the pom of mevan,i just add two dependency in IED's auto generated POM.
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>com.netflix.ribbon</groupId>
<artifactId>ribbon</artifactId>
<version>2.2.2</version>
</dependency>
the code for printing src_port
#Sharable
public class InHandle extends ChannelInboundHandlerAdapter {
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(ctx.channel().localAddress());
super.channelRead(ctx, msg);
}
}
public class Pipeline implements PipelineConfigurator<ByteBuf, ByteBuf> {
public InHandle handler;
Pipeline() {
handler = new InHandle();
}
public void configureNewPipeline(ChannelPipeline pipeline) {
pipeline.addFirst(handler);
}
}
and change the client = RibbonTransport.newTcpClient(lb, impl);to Pipeline pipe = new Pipeline();client = RibbonTransport.newTcpClient(lb, pipe, impl, new DefaultLoadBalancerRetryHandler(impl));

So, your App() constructor does the initialization of lb/client/etc.
Then you're starting 40 different threads with 40 different RxClient instances (each instance has own pool by default) by calling new App() in the first for loop. To make things clear - the way you spawn multiple RxClient instances here does not allow them to share any common pool. Try to use one RxClient instance instead.
What if you change your main method like below, does it stop creating extra sockets?
public static void main( String[] args ) throws Exception
{
App app = new App() // Create things just once
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(()->app.run()); // pass the run()
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
If above does not help fully (at least it will reduce created sockets count in 40 times) - can you please clarify how exactly do you determine that:
i find it will create a new socket everytime i call sendAndRecvByRibbon
and what are your measurements after you update constructor with this line:
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
impl.set(CommonClientConfigKey.PoolMaxThreads,1); //Add this one and test
Update
Yes, looking at the sendAndRecvByRibbon it seems that it lacks marking the PooledConnection as no longer acquired by calling close once you don't expect any further reads from it.
As long as you expect the only single read event, just change this line
connection.getInput()
to the
return connection.getInput().zipWith(Observable.just(connection), new Func2<ByteBuf, ObservableConnection<ByteBuf, ByteBuf>, ByteBuf>() {
#Override
public ByteBuf call(ByteBuf byteBuf, ObservableConnection<ByteBuf, ByteBuf> conn) {
conn.close();
return byteBuf;
}
});
Note, that if you'd design more complex protocol over TCP, then input bytebuf can be analyzed for your specific 'end of communication' sign which indicates the connection can be returned to the pool.

Producer and Client Connection

I have am working on a small project that will create a connection between a Producer and a Client. The producer will take an email object, put it in a queue, and the client will pick it up. I have two questions about this. There is a separate Email class not shown which just has one getter and setter method for a local String variable called content.
1) Why am I getting a null pointer exception in the line p.sendMessage(email), even though all the parameters are correct?
2) After the producer program has ended how will the values inside of the queue be stored so I can pick them up from the Client. Am I supposed to use some sort of server or database?
public class Producer {
public static Queue que;
public static void main(String[] args) {
Producer p = new Producer();
Email email = new Email("Content of email");
p.sendMessage(email);
}
public void sendMessage(Email e){
Gson gson = new GsonBuilder().create();
String read = (gson.toJson(e));
que.add(read);
}
}
public class Client {
public static void main(String[] args) {
Client c = new Client();
c.pickMessage();
}
public void pickMessage(){
String s = Producer.que.remove().toString();
Gson gson = new GsonBuilder().create();
Email newEmail = gson.fromJson(s,Email.class);
System.out.println(newEmail.getEmailContent());
}
}

How to use Authentication with JIRA REST API in Java

hi im creating a simple tool using java to create,update and delete issues(tickets) in jira. i am using rest api following code is im using to authenticate jira and issue tickets.
public class JiraConnection {
public static URI jiraServerUri = URI.create("http://localhost:8090/jira/rest/api/2/issue/HSP-1/");
public static void main(String args[]) throws IOException {
final AsynchronousJiraRestClientFactory factory = new AsynchronousJiraRestClientFactory();
final JiraRestClient restClient = factory.createWithBasicHttpAuthentication(jiraServerUri,"vinuvish92#gmail.com","vinu1994");
System.out.println("Sending issue creation requests...");
try {
final List<Promise<BasicIssue>> promises = Lists.newArrayList();
final IssueRestClient issueClient = restClient.getIssueClient();
System.out.println("Sending issue creation requests...");
for (int i = 0; i < 100; i++) {
final String summary = "NewIssue#" + i;
final IssueInput newIssue = new IssueInputBuilder("TST", 1L, summary).build();
System.out.println("\tCreating: " + summary);
promises.add(issueClient.createIssue(newIssue));
}
System.out.println("Collecting responses...");
final Iterable<BasicIssue> createdIssues = transform(promises, new Function<Promise<BasicIssue>, BasicIssue>() {
#Override
public BasicIssue apply(Promise<BasicIssue> promise) {
return promise.claim();
}
});
System.out.println("Created issues:\n" + Joiner.on("\n").join(createdIssues));
} finally {
restClient.close();
}
}
}
according this code i couldn't connect to the jira
**following exception i am getting **
please suggest me best solution to do my task

It seems to me that your error is clearly related to url parameter. The incriminated line and the fact that the error message is about not finding the resource are good indications of it.
You don't need to input the whole endpoint since you are using the JiraRestClient. Depending on the method that you call it will resolve the endpoint. Here is an example that works: as you can see I only input the base url

What security issues come from calling methods with reflection?

I'm working on a project that has hosts and clients, and where hosts can send commands to clients (via sockets).
I'm determined that using JSON to communicate works the best.
For example:
{
"method" : "toasty",
"params" : ["hello world", true]
}
In this example, when this JSON string is sent to the client, it will be processed and a suitable method within the client will be run as such:
public abstract class ClientProcessor {
public abstract void toasty(String s, boolean bool);
public abstract void shutdown(int timer);
private Method[] methods = getClass().getDeclaredMethods();
public void process(String data) {
try {
JSONObject json = new JSONObject(data);
String methodName = (String) json.get("method");
if (methodName.equals("process"))
return;
for (int i = 0; i < methods.length; i++)
if (methods[i].getName().equals(methodName)) {
JSONArray arr = json.getJSONArray("params");
int length = arr.length();
Object[] args = new Object[length];
for (int i2 = 0; i2 < length; i2++)
args[i2] = arr.get(i2);
methods[i].invoke(this, args);
return;
}
} catch (Exception e) {}
}
}
And using the ClientProcessor:
public class Client extends ClientProcessor {
#Override
public void toasty(String s, boolean bool) {
//make toast here
}
#Override
public void shutdown(int timer) {
//shutdown system within timer
}
public void processJSON(String json) {
process(json);
}
}
The JSON is sent by the server to the client, but the server could be modified to send different JSONs.
My questions are:
Is this a safe way of running methods by processing JSON?
Is there a better way to do this? I'm thinking that using reflection is terribly slow.

There's a 100 and 1 ways you can process a JSON message so that some processing occurs, but they'll all boil down to:
parse message
map message to method
invoke method
send response
While you could use a reflective call (performance-wise it would be fine for most cases) to invoke a method, that, imho, would be a little too open - a malicious client could for example crash your system by issuing wait calls.
Reflection also opens you up to having to correctly map the parameters, which is more complicated than the code you've shown in your question.
So don't use Reflection.
Would you could do is define a simple interface, implementations of which would understand how to process the parameters and have your processor (more commonly referred to as a Controller) invoke that, something like this:
public interface ServiceCall
{
public JsonObject invoke(JsonArray params) throws ServiceCallException;
}
public class ServiceProcessor
{
private static final Map<String, ServiceCall> SERVICE_CALLS = new HashMap<>();
static
{
SERVICE_CALLS.put("toasty", new ToastCall());
}
public String process(String messageStr)
{
try
{
JsonObject message = Json.createReader(new StringReader(messageStr)).readObject();
if (message.containsKey("method"))
{
String method = message.getString("method");
ServiceCall serviceCall = SERVICE_CALLS.get(method);
if (serviceCall != null)
{
return serviceCall.invoke(message.getJsonArray("params")).toString();
}
else
{
return fail("Unknown method: " + method);
}
}
else
{
return fail("Invalid message: no method specified");
}
}
catch (Exception e)
{
return fail(e.message);
}
}
private String fail(String message)
{
return Json.createObjectBuilder()
.add("status", "failed")
.add("message", message)
.build()
.toString();
}
private static class ToastCall implements ServiceCall
{
public JsonObject invoke(JsonArray params) throws ServiceCallException
{
//make toast here
}
}
}

Map method names to int constants and just switch(case) on these constants to invoke appropriate method.
"toasty" : 1
"shutdown": 2
switch()
case 1: toasty()
case 2: shutdown()

I believe you are trying to convert JSON string to Java object and vice versa... if that is the requirement then this would not be the right approach...
Try any open source API like Gson...
it is the API by Google for conversin of Java to JSON and vice versa.
Please check ...
https://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/Gson.html
Let me know if you have any further questions...

Read an AMF object with flex socket

I'm currently trying to communicate between java and flex by using sockets and AMF serialized objects.
On the java side I use Amf3Input and Amf3Output from BlazeDS (flex-messaging-common.jar and flex-messaging-core.jar).
The connection is correctly established, and if i try to send object from flex to java, i can easily read objects :
FLEX side :
protected function button2_clickHandler(event:MouseEvent):void
{
var tmp:FlexAck = new FlexAck;
tmp.id="123456789123456789123456789";
tmp.name="A";
tmp.source="Aaaaaa";
tmp.ackGroup=false;
s.writeObject(tmp);
s.flush();
}
JAVA side :
ServerSocket servSoc = new ServerSocket(8888);
Socket s = servSoc.accept();
Amf3Output amf3Output = new Amf3Output(SerializationContext.getSerializationContext());
amf3Output.setOutputStream(s.getOutputStream());
Amf3Input amf3Input = new Amf3Input(SerializationContext.getSerializationContext());
amf3Input.setInputStream(s.getInputStream());
while(true)
{
try
{
Object obj = amf3Input.readObject();
if(obj!=null){
if (obj instanceof AckOrder){
System.out.println(((AckOrder)obj).getId());
}
}
}
catch (Exception e)
{
e.printStackTrace();
break;
}
}
amf3Output.close();
amf3Input.close();
servSoc.close();
In this way it works perfectly, but the problem is to read objects sent from the java side.
The code I use in java is :
for(int i=0;i<10;i++){
ack = new AckOrder(i,"A","B", true);
amf3Output.writeObject(ack);
amf3Output.writeObjectEnd();
amf3Output.flush();
}
I have an handler on ProgressEvent.SOCKET_DATA :
trace((s.readObject() as FlexAck).id);
But I have errors such as :
Error #2030: End of File detected
Error #2006: Index Out of bound
If i add manipulations on ByteArrays, i manage to read the first object, but not the following.
s.readBytes(tmp,tmp.length);
content = clone(tmp);
(content.readObject());
trace("########################## OK OBJECT RECEIVED");
var ack:FlexAck = (tmp.readObject() as FlexAck);
trace("**********************> id = "+ack.id);
I've spent many our trying to find something in several forums etc, but nothing helped.
So if someone could help me it would be great.
Thanks
Sylvain
EDIT :
Here is an example that I thought should work, but doesn't I hope that it's better illustrate what I aim to do (permanent connection with socket and an exchange of messages).
Java class :
import java.net.ServerSocket;
import java.net.Socket;
import awl.oscare.protocol.AckOrder;
import flex.messaging.io.SerializationContext;
import flex.messaging.io.amf.Amf3Input;
import flex.messaging.io.amf.Amf3Output;
public class Main {
public static void main(String[] args) {
while(true)
{
try {
ServerSocket servSoc = new ServerSocket(8888);
Socket s = servSoc.accept();
System.out.println("connection accepted");
Amf3Output amf3Output = new Amf3Output(SerializationContext.getSerializationContext());
amf3Output.setOutputStream(s.getOutputStream());
Amf3Input amf3Input = new Amf3Input(SerializationContext.getSerializationContext());
amf3Input.setInputStream(s.getInputStream());
while(true)
{
try
{
System.out.println("Reading object");
Object obj = amf3Input.readObject();
if(obj!=null)
{
System.out.println(obj.getClass());
if (obj instanceof AckOrder)
{
AckOrder order = new AckOrder();
order.setId(((AckOrder)obj).getId());
order.setName(((AckOrder)obj).getName());
order.setSource(((AckOrder)obj).getSource());
order.setAckGroup(((AckOrder)obj).isAckGroup());
System.out.println(((AckOrder)obj).getId());
amf3Output.writeObject(order);
amf3Output.writeObjectEnd();
amf3Output.flush();
}
}
}
catch (Exception e)
{
e.printStackTrace();
break;
}
}
amf3Output.close();
amf3Input.close();
servSoc.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Java Serializable object :
package protocol;
import java.io.Serializable;
public class AckOrder implements Serializable {
private static final long serialVersionUID = 5106528318894546695L;
private String id;
private String name;
private String source;
private boolean ackGroup = false;
public String getId() {
return this.id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public void setSource(String source) {
this.source = source;
}
public String getSource() {
return this.source;
}
public void setAckGroup(boolean ackGroup) {
this.ackGroup = ackGroup;
}
public boolean isAckGroup() {
return this.ackGroup;
}
public AckOrder()
{
super();
}
}
Flex Side :
Main flex code :
<fx:Script>
<![CDATA[
import mx.collections.ArrayCollection;
import mx.controls.Alert;
import mx.events.FlexEvent;
import mx.utils.object_proxy;
private var _socket:Socket = new Socket();;
private function onCreationComplete():void
{
this._socket.connect("localhost",8888);
this._socket.addEventListener(ProgressEvent.SOCKET_DATA, onData);
}
private function onData(e:ProgressEvent):void
{
if(this._socket.bytesAvailable)
{
this._socket.endian = Endian.LITTLE_ENDIAN;
var objects:Array = [];
try{
while(this._socket.bytesAvailable > 0)
{
objects.push(this._socket.readObject());
}
}catch(e:Error){trace(e.message);}
trace("|"+(objects)+"|");
}
}
protected function sendButton_clickHandler(event:MouseEvent):void
{
var tmp:FlexAck = new FlexAck;
tmp.id="1";
tmp.name="A";
tmp.source="B";
tmp.ackGroup=false;
this._socket.writeObject(tmp);
this._socket.flush();
}
]]>
</fx:Script>
<s:Button x="0" y="0" name="send" label="Send" click="sendButton_clickHandler(event)"/>
Flex serializable object :
package
{
[Bindable]
[RemoteClass(alias="protocol.AckOrder")]
public class FlexAck
{
public function FlexAck()
{
}
public var id:String;
public var name:String;
public var source:String;
public var ackGroup:Boolean;
}
}
Edit 25/05/2011 :
I've added those listeners in my flex code :
this._socket.addEventListener(Event.ACTIVATE,onActivate);
this._socket.addEventListener(Event.CLOSE,onClose);
this._socket.addEventListener(Event.CONNECT,onConnect);
this._socket.addEventListener(Event.DEACTIVATE,onDeactivate);
this._socket.addEventListener(IOErrorEvent.IO_ERROR,onIOerror);
this._socket.addEventListener(SecurityErrorEvent.SECURITY_ERROR,onSecurityError);
But There's no errors and I still don't manage to receive objects correctly.

You have to send the AMF data as ByteArray on the server:
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
amf3Output.setOutputStream(baos);
amf3Output.writeObject(order);
amf3Output.flush();
amf3Output.close();
s.getOutputStream().write(baos.toByteArray());
Then
this._socket.readObject()
works as expected !

Hi the problem is caused by the following:
An AMF stream is stateful. When it serializes objects, it compresses them relative to objects that it have already been written.
Compression is achieved by referencing previously sent class descriptions, string values and objects using indexes (so for example, if the first string you sent was "heloWorld", when you later send that string, the AMF stream will sent string index 0).
Unfortunately, ByteArray and Socket do not maintain reference tables between readObject calls. Thus, even if you keep appending your newly read objects to the end of the same ByteArray object, each call to readObject instantiates new reference tables, discarding previously created ones (this means it should work for repeated references to the same string within an object tree)
In your example, you are always writing the same string values to properties. Thus when you send the second object, its string properties are not serialized as strings, but as references to the strings in the previously written object.
The solution, is to create a new AMF stream for each object you send.
This is complete rubbish of course(!) It means we can't really utilize the compression in custom protocols. It would be much better if our protocols could decide when to reset the these reference tables, perhaps when they got too big.
For example, if you have an RPC protocol, it would be nice to have an AMF stream pass the remote method names as references rather than strings for speed...
I haven't checked but I think this sort of thing is done by RTMP. The reason it probably wouldn't have been made available in developer objects like ByteArray and Socket (sigh, I hope this isn't true) is because Adobe wants to push us towards LCDS...
Addendum/edit: just found this, which provides a solution http://code.google.com/p/cvlib/

After looking at the code, I think what you want to do on the Java end is this:
for(int i=0;i<10;i++){
ack = new AckOrder(i,"A","B", true);
amf3Output.writeObject(ack);
}
amf3Output.flush();
When you do 'flush', you're sending information over the socket so you only had one object being sent at a time. On the Flex end, you should always try to see what's the length of the object and make sure you're not going over it which would cause this error.
EDIT:
private var _socket:Socket = new Socket();
private function onCreationComplete():void
{
// Add connection socket info here
this._socket.addEventListener(ProgressEvent.SOCKET_DATA, onData);
}
// This gets called every time we get new info, as in after the server flushes
private function onData(e:ProgressEvent):void
{
if(this._socket.bytesAvailable)
{
this._socket.endian = Endian.LITTLE_ENDIAN; // Might not be needed, but often is
// Try to get objects
var objects:Array = [];
try{
while(this._socket.bytesAvailable > 0)
{
objects.push(this._socket.readObject());
}
}catch(e:Error){}
// Do something with objects array
}
}
The onData function is called continually (every time the server sends info) since everything is asynchronous.