Develop Reference

Resetting OutputStream so updated variables send

I've come across an issue when sending an object over a local connection. The object will send the first time as expected. However variables of the object are constantly being changed therefore need to be updated when sending to the other connection. This is done through sending messages prompting the other client to listen and wait for the object being sent.
I'm aware that the java.io.ObjectOutputStream.reset() method exists but keep getting the following error:
error: cannot find symbol
output.reset();
Here's how the code is currently structured (Minus lots of non relevant code):
import java.net.*;
import java.util.*
import java.io.*;
public class Client
{
private static Socket cSocket = null;
private static ObjectOutput output = null;
private static Person myPerson = null;
private static String serverHost = "localhost";
public void Run()
{
// Declaring the output
output = new ObjectOutputStream(
cSocket.getOutputStream()
);
}
private static void sendPerson()
{
try
{
output.writeObject( myPerson );
output.reset();
} catch (Exception e)
{
}
}
}
TLDR: Each time sendPerson() is called the other client receives the first object sent to the other client rather than the updated variables. Tried using reset(); but error is thrown.
Would just like the objects updated variables to be sent rather than the initial object always being sent.

To use ObjectOutputStream#reset you have to define your field with such type.
Not ObjectOutput, but ObjectOutputStream.
Like this:
private static ObjectOutputStream output = null;

RxNetty reuse the connection

I want to use Netflix-Ribbon as TCP client load balancer without Spring Cloud,and i write test code.
public class App implements Runnable
{
public static String msg = "hello world";
public BaseLoadBalancer lb;
public RxClient<ByteBuf, ByteBuf > client;
public Server echo;
App(){
lb = new BaseLoadBalancer();
echo = new Server("localhost", 8000);
lb.setServersList(Lists.newArrayList(echo));
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
client = RibbonTransport.newTcpClient(lb, impl);
}
public static void main( String[] args ) throws Exception
{
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(new App());
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
public String sendAndRecvByRibbon(final String data)
{
String response = "";
try {
response = client.connect().flatMap(new Func1<ObservableConnection<ByteBuf, ByteBuf>,
Observable<ByteBuf>>() {
public Observable<ByteBuf> call(ObservableConnection<ByteBuf, ByteBuf> connection) {
connection.writeStringAndFlush(data);
return connection.getInput();
}
}).timeout(1, TimeUnit.SECONDS).retry(1).take(1)
.map(new Func1<ByteBuf, String>() {
public String call(ByteBuf ByteBuf) {
return ByteBuf.toString(Charset.defaultCharset());
}
})
.toBlocking()
.first();
}
catch (Exception e) {
System.out.println(((LoadBalancingRxClientWithPoolOptions) client).getMaxConcurrentRequests());
System.out.println(lb.getLoadBalancerStats());
}
return response;
}
public void run() {
for (int i = 0; i < 200; i++) {
sendAndRecvByRibbon(msg);
}
}
}
i find it will create a new socket everytime i callsendAndRecvByRibbon even though the poolEnabled is setting to true. So,it confuse me,i miss something?
and there are no option to configure the size of the pool,but hava a PoolMaxThreads and MaxConnectionsPerHost.
My question is how to use a connection pool in my simple code, and what's wrong with my sendAndRecvByRibbon,it open a socket then use it only once,how can i reuse the connection?thanks for your time.
the server is just a simple echo server writing in pyhton3,i comment outconn.close() because i want to use long connection.
import socket
import threading
import time
import socketserver
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
conn = self.request
while True:
client_data = conn.recv(1024)
if not client_data:
time.sleep(5)
conn.sendall(client_data)
# conn.close()
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
if __name__ == "__main__":
HOST, PORT = "localhost", 8000
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
server.serve_forever()
and the pom of mevan,i just add two dependency in IED's auto generated POM.
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>com.netflix.ribbon</groupId>
<artifactId>ribbon</artifactId>
<version>2.2.2</version>
</dependency>
the code for printing src_port
#Sharable
public class InHandle extends ChannelInboundHandlerAdapter {
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(ctx.channel().localAddress());
super.channelRead(ctx, msg);
}
}
public class Pipeline implements PipelineConfigurator<ByteBuf, ByteBuf> {
public InHandle handler;
Pipeline() {
handler = new InHandle();
}
public void configureNewPipeline(ChannelPipeline pipeline) {
pipeline.addFirst(handler);
}
}
and change the client = RibbonTransport.newTcpClient(lb, impl);to Pipeline pipe = new Pipeline();client = RibbonTransport.newTcpClient(lb, pipe, impl, new DefaultLoadBalancerRetryHandler(impl));

So, your App() constructor does the initialization of lb/client/etc.
Then you're starting 40 different threads with 40 different RxClient instances (each instance has own pool by default) by calling new App() in the first for loop. To make things clear - the way you spawn multiple RxClient instances here does not allow them to share any common pool. Try to use one RxClient instance instead.
What if you change your main method like below, does it stop creating extra sockets?
public static void main( String[] args ) throws Exception
{
App app = new App() // Create things just once
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(()->app.run()); // pass the run()
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
If above does not help fully (at least it will reduce created sockets count in 40 times) - can you please clarify how exactly do you determine that:
i find it will create a new socket everytime i call sendAndRecvByRibbon
and what are your measurements after you update constructor with this line:
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
impl.set(CommonClientConfigKey.PoolMaxThreads,1); //Add this one and test
Update
Yes, looking at the sendAndRecvByRibbon it seems that it lacks marking the PooledConnection as no longer acquired by calling close once you don't expect any further reads from it.
As long as you expect the only single read event, just change this line
connection.getInput()
to the
return connection.getInput().zipWith(Observable.just(connection), new Func2<ByteBuf, ObservableConnection<ByteBuf, ByteBuf>, ByteBuf>() {
#Override
public ByteBuf call(ByteBuf byteBuf, ObservableConnection<ByteBuf, ByteBuf> conn) {
conn.close();
return byteBuf;
}
});
Note, that if you'd design more complex protocol over TCP, then input bytebuf can be analyzed for your specific 'end of communication' sign which indicates the connection can be returned to the pool.

Java Socket: receiving wrong boolean

I have a client/server architecture that sends class-instances via sockets.
In one class I have a boolean:
public class Survey implements Serializable {
private static final long serialVersionUID = -1156493488498723461L;
private boolean isExpired;
public Survey() {
this.isExpired = false;
}
public void markAsExpired() {
this.isExpired = true;
}
public boolean isExpired() {
return isExpired;
}
}
The part where I send the packet:
survey.markAsExpired();
HashMap<Header, Object> packet = new HashMap<Header, Object>();
packet.put(header, survey);
System.out.println(survey.isExpired()); // prints true
try {
socketOutput.writeObject(packet);
socketOutput.flush();
} catch (IOException e) {
e.printStackTrace();
}
When I send that class with the boolean set to true (see above), the client always receives it as false.
Where does this come from?

I could solve the problem by sending a deep copy instead of the object itself. Workaround but worked.

What security issues come from calling methods with reflection?

I'm working on a project that has hosts and clients, and where hosts can send commands to clients (via sockets).
I'm determined that using JSON to communicate works the best.
For example:
{
"method" : "toasty",
"params" : ["hello world", true]
}
In this example, when this JSON string is sent to the client, it will be processed and a suitable method within the client will be run as such:
public abstract class ClientProcessor {
public abstract void toasty(String s, boolean bool);
public abstract void shutdown(int timer);
private Method[] methods = getClass().getDeclaredMethods();
public void process(String data) {
try {
JSONObject json = new JSONObject(data);
String methodName = (String) json.get("method");
if (methodName.equals("process"))
return;
for (int i = 0; i < methods.length; i++)
if (methods[i].getName().equals(methodName)) {
JSONArray arr = json.getJSONArray("params");
int length = arr.length();
Object[] args = new Object[length];
for (int i2 = 0; i2 < length; i2++)
args[i2] = arr.get(i2);
methods[i].invoke(this, args);
return;
}
} catch (Exception e) {}
}
}
And using the ClientProcessor:
public class Client extends ClientProcessor {
#Override
public void toasty(String s, boolean bool) {
//make toast here
}
#Override
public void shutdown(int timer) {
//shutdown system within timer
}
public void processJSON(String json) {
process(json);
}
}
The JSON is sent by the server to the client, but the server could be modified to send different JSONs.
My questions are:
Is this a safe way of running methods by processing JSON?
Is there a better way to do this? I'm thinking that using reflection is terribly slow.

There's a 100 and 1 ways you can process a JSON message so that some processing occurs, but they'll all boil down to:
parse message
map message to method
invoke method
send response
While you could use a reflective call (performance-wise it would be fine for most cases) to invoke a method, that, imho, would be a little too open - a malicious client could for example crash your system by issuing wait calls.
Reflection also opens you up to having to correctly map the parameters, which is more complicated than the code you've shown in your question.
So don't use Reflection.
Would you could do is define a simple interface, implementations of which would understand how to process the parameters and have your processor (more commonly referred to as a Controller) invoke that, something like this:
public interface ServiceCall
{
public JsonObject invoke(JsonArray params) throws ServiceCallException;
}
public class ServiceProcessor
{
private static final Map<String, ServiceCall> SERVICE_CALLS = new HashMap<>();
static
{
SERVICE_CALLS.put("toasty", new ToastCall());
}
public String process(String messageStr)
{
try
{
JsonObject message = Json.createReader(new StringReader(messageStr)).readObject();
if (message.containsKey("method"))
{
String method = message.getString("method");
ServiceCall serviceCall = SERVICE_CALLS.get(method);
if (serviceCall != null)
{
return serviceCall.invoke(message.getJsonArray("params")).toString();
}
else
{
return fail("Unknown method: " + method);
}
}
else
{
return fail("Invalid message: no method specified");
}
}
catch (Exception e)
{
return fail(e.message);
}
}
private String fail(String message)
{
return Json.createObjectBuilder()
.add("status", "failed")
.add("message", message)
.build()
.toString();
}
private static class ToastCall implements ServiceCall
{
public JsonObject invoke(JsonArray params) throws ServiceCallException
{
//make toast here
}
}
}

Map method names to int constants and just switch(case) on these constants to invoke appropriate method.
"toasty" : 1
"shutdown": 2
switch()
case 1: toasty()
case 2: shutdown()

I believe you are trying to convert JSON string to Java object and vice versa... if that is the requirement then this would not be the right approach...
Try any open source API like Gson...
it is the API by Google for conversin of Java to JSON and vice versa.
Please check ...
https://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/Gson.html
Let me know if you have any further questions...

Java JAX-RS web service: adding nodes to JAXB XML result as threads complete

I have programmed a JAX-RS web service with Jersey that queries prices from different websites and gives the result back as XML through JAXB annotated classes. Unfortunately some websites take up to 15 seconds to respond so I am using multiple threads to inquire those prices.
I would like to write a client to this webservice now and my web users will not want to wait for 30 seconds after they hit 'search' for the result to come so my idea is dynamically updating the result table as the results from my JAX-RS webservice come back.
After 30 seconds my webservice should time out and close the <result>-Element or after all threads completed.
Right now my webservice runs all threads and gives back the result after all trheads are completed, I would like to dynamically add results to the XML output as they come, how can I accomplish that?
The structure of the XML response is:
<result>
<articles>
<article>
content of article
</article>
</articles>
As the webservice gets results from websites it adds new articles to the XML
</result>
RequestController.java
#Path("/request")
public class RequestController {
#GET
#Produces("application/xml")
public Response getRequest(#QueryParam("part") String part) {
response = new Response();
driverController = new DriverController(this.response, this.part);
this.response = driverController.query();
return this.response;
}
}
DriverController.java
public class DriverController {
public Response query() {
CompletionService<Deque<Article>> completionService = new ExecutorCompletionService<Deque<Article>>(
Worker.getThreadPool());
final Deque<Article> articleQueue = new LinkedList<Article>();
int submittedTasks = 0;
// This threadwill take about 4 seconds to finish
Driver driverA = new DriverA(this.part,
this.currency, this.language);
// This thread will take about 15 seconds to finish
Driver driverN = new DriverN(this.part,
this.currency, this.language);
completionService.submit(driverA);
submittedTasks++;
completionService.submit(driverN);
submittedTasks++;
for (int i = 0; i < submittedTasks; i++) {
log.info("Tasks: " + submittedTasks);
try {
Future<Deque<Article>> completedFuture = completionService.take();
try {
Deque<Article> articleQueueFromThread = completedFuture.get();
if (articleQueueFromThread != null) {
articleQueue.addAll(articleQueueFromThread);
response.setStatus("OK");
}
} catch (ExecutionException e) {
log.error(e.getMessage());
e.printStackTrace();
}
} catch (InterruptedException e) {
log.error(e.getMessage());
e.printStackTrace();
}
}
for (Article article : articleQueue) {
this.response.addArticle(article);
}
return this.response;
}
}
Response.java
#XmlRootElement
public class Response {
Queue<Article> queue = new ConcurrentLinkedQueue<Article>();
private String status;
private String code;
private String message;
private List<Article> articles = new ArrayList<Article>();
public Response(){
}
public void setMessage(String message) {
this.message = message;
}
#XmlAttribute
public String getMessage() {
return message;
}
public void setStatus(String status) {
this.status = status;
}
#XmlAttribute
public String getStatus() {
return status;
}
public void setCode(String code) {
this.code = code;
}
#XmlAttribute
public String getCode() {
return code;
}
public void addArticle(Article article) {
this.articles.add(article);
System.out.println("Response: ADDED ARTICLE TO RESPONSE");
}
#XmlElement(name = "article")
#XmlElementWrapper(name = "articles")
public List<Article> getArticles() {
return articles;
}
}

I started to adapt your code to do it, but I decided it was easier to work up an independent example. The example starts a Grizzly+Jersey server with a single resource class in it. A GET on the resource spawns three threads that delay for 2, 4, and 6 seconds before returning some objects. After the server starts, another thread makes a request to the server. When you run it, you can plainly see that the requester receives chunks of XML as the respective threads finish their work in the server. The one thing it doesn't do is wrap separately-delivered XML chunks in a single root element since that should be relatively trivial.
The entire executable source is below, and if you have maven and git, you can clone it from github and run it with:
git clone git://github.com/zzantozz/testbed.git tmp
cd tmp
mvn compile exec:java -Dexec.mainClass=rds.jersey.JaxRsResource -pl jersey-with-streaming-xml-response
Source:
import com.sun.grizzly.http.SelectorThread;
import com.sun.jersey.api.container.grizzly.GrizzlyWebContainerFactory;
import javax.ws.rs.*;
import javax.ws.rs.core.StreamingOutput;
import javax.xml.bind.*;
import javax.xml.bind.annotation.*;
import java.io.*;
import java.net.*;
import java.util.*;
import java.util.concurrent.*;
#Path("/streaming")
public class JaxRsResource {
private static ExecutorService executorService = Executors.newFixedThreadPool(4);
private static int fooCounter;
private Marshaller marshaller;
public JaxRsResource() throws JAXBException {
marshaller = JAXBContext.newInstance(Foo.class).createMarshaller();
marshaller.setProperty("jaxb.fragment", Boolean.TRUE);
}
#GET
#Produces("application/xml")
public StreamingOutput streamStuff() {
System.out.println("Got request for streaming resource; starting delayed response threads");
final List<Future<List<Foo>>> futureFoos = new ArrayList<Future<List<Foo>>>();
futureFoos.add(executorService.submit(new DelayedFoos(2)));
futureFoos.add(executorService.submit(new DelayedFoos(4)));
futureFoos.add(executorService.submit(new DelayedFoos(6)));
return new StreamingOutput() {
public void write(OutputStream output) throws IOException {
for (Future<List<Foo>> futureFoo : futureFoos) {
writePartialOutput(futureFoo, output);
output.write("\n".getBytes());
output.flush();
}
}
};
}
private void writePartialOutput(Future<List<Foo>> futureFoo, OutputStream output) {
try {
List<Foo> foos = futureFoo.get();
System.out.println("Server sending a chunk of XML");
for (Foo foo : foos) {
marshaller.marshal(foo, output);
}
} catch (JAXBException e) {
throw new IllegalStateException("JAXB couldn't marshal. Handle it.", e);
} catch (InterruptedException e) {
throw new IllegalStateException("Task was interrupted. Handle it.", e);
} catch (ExecutionException e) {
throw new IllegalStateException("Task failed to execute. Handle it.", e);
}
}
class DelayedFoos implements Callable<List<Foo>> {
private int delaySeconds;
public DelayedFoos(int delaySeconds) {
this.delaySeconds = delaySeconds;
}
public List<Foo> call() throws Exception {
Thread.sleep(delaySeconds * 1000);
return Arrays.asList(new Foo(fooCounter++), new Foo(fooCounter++), new Foo(fooCounter++));
}
}
public static void main(String[] args) throws IOException {
System.out.println("Starting Grizzly with the JAX-RS resource");
final String baseUri = "http://localhost:9998/";
final Map<String, String> initParams = new HashMap<String, String>();
initParams.put("com.sun.jersey.config.property.packages", "rds.jersey");
SelectorThread threadSelector = GrizzlyWebContainerFactory.create(baseUri, initParams);
System.out.println("Grizzly started");
System.out.println("Starting a thread to request the streamed XML");
executorService.submit(new HttpRequester(baseUri + "streaming"));
}
}
#XmlRootElement
class Foo {
#XmlElement
private int id;
Foo() {}
public Foo(int id) {
this.id = id;
}
}
class HttpRequester implements Runnable {
private String url;
public HttpRequester(String url) {
this.url = url;
}
public void run() {
try {
System.out.println("Doing HTTP GET on " + url);
HttpURLConnection urlConnection = (HttpURLConnection) new URL(url).openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("Client got: " + line);
}
System.exit(0);
} catch (IOException e) {
throw new IllegalStateException("Some bad I/O happened. Handle it.", e);
}
}
}
Important points/differences to take note of:
Returning a Response from your resource method indicates that the entire response is contained in that object and doesn't allow for incremental updates to the response. Return a StreamingOutput instead. That tells Jersey that you'll be sending back a stream of data, which you can append to at will until you're done. The StreamingOutput gives you access to an OutputStream, which is what you use to send incremental updates and is the key to this whole thing. Of course, that means you have to handle the marshaling yourself. Jersey can only do the marshaling if you're returning the entire response at once.
Since the OutputStream is how you send back the data a little at a time, you either have to do the threading in your JAX-RS resource or pass the OutputStream down to your DriverController and write to it there.
Be sure to invoke flush() on the OutputStream if you want to force it to send out data immediately. Otherwise, nothing will be sent to the client until whatever internal buffer is filled up. Note that invoking flush() yourself circumvents the purpose of the buffer and makes your app more chatty.
All in all, to apply this to your project, the primary thing to do is change your resource method to return a StreamingOutput implementation and invoke your DriverController from inside that implementation, passing the OutputStream to the DriverController. Then in the DriverController, when you get some Articles back from a thread, instead of adding it to a queue for later, write it to the OutputStream immediately.

#Ryan Stewart: how would we resolve same issue in axis2.x SOAP based web service kind of environment and HTML page as web client.
What I think is DriverController can keep Future objects in session and returns very first available response(article) with a unique session identifier to client....then client can make another webservice call (preferably thru Ajax+jquery) passing saved session identifier which would trigger DriverController to search more results and send back....is it a viable solution? Would it applicable for above environment too.