I have two applications, one that sends UDP messages using Camel with the Netty component, and one that receives UDP messages in Java NIO with DatagramChannel.
When receiving the data, I've noticed that there's an extra 29 bytes prepended to the front of my message. Netty Camel prints out the outgoing bytes and it looks fine, but when I do a packet.getData() as soon as the message comes in on the other side, it has extra stuff on the front (and it's always the same bytes).
Is Camel or Netty wrapping the packet before sending it?
[edit] Additional information:
-Camel is printing the log statement, not Netty
-the bytes prepended to the message change when the content of the message changes (only two bytes are changed)
I know this question is pretty old now, but I hit exactly this problem and it took me a while to find the solution. So here it is...
Basically the problem boils down to confusion of what camel-netty will do when you're telling it to send something sufficiently low-level like a byte[] in a UDP packet. I expect that like me the OP assumed they were setting raw data, but camel-netty uses Java Object Serialization by default - resulting in those extra "random" bytes appearing before the expected data.
The solution is to change the encoder/decoder used by the endpoint(s) in question. There are various built-in alternatives, but you can subclass them if you need something more... weird. Either way, the process is:
1) Add the "encoder=#myEncoder" and "decoder=#myDecoder" options as appropriate on to the endpoint URIs. e.g.
String destinationUri = "netty:udp://localhost:4242"
+ "?sync=false&encoder=#myEncoder";
2) Add a mapping from "myEncoder" to an instance of your new Encoder class to a Camel Registry. Same for myDecoder. Then use that registry when constructing the CamelContext. e.g.
SimpleRegistry registry = new SimpleRegistry();
registry.put("myEncoder", new StringEncoder());
registry.put("myDecoder", new StringDecoder());
CamelContext camelContext = new CamelContext(registry);
Obviously the real trick is in finding or making an Encoder/Decoder that suits your needs. A blog post at znetdevelopment really helped me, though it goes one step further and puts the custom Encoders in a custom Pipeline (I ignored that bit).
Related
I have implemented a TCP socket using JAVA and Netty where I am able to receive Strings from clients.
Now I also have reqirements to accept XML based strings and not quite sure which kind of delimiter I could use. At the moment I have the following implementation.
pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
With this the XML is read in line by line which is obvious. Now how can I tell Netty to read the whole XML string in? Am looking for some sort of xmlDelimiter.
Could someone help me out?
Thanks
This is a protocol issue that both sender and receiver has to agree upon. A real life example is that of NETCONF (used to configure network devices). NETCONF uses XML for both protocol operations and data encoding, and needed a mechanism for delimiting individual XML messages. The first version of the specification defined the string ]]>]]> as the delimiter. A sequence of XML messages could looke like so:
<rpc message-id="1"> ... </rpc>
]]>]]>
<rpc message-id="2"> ... </rpc>
]]>]]>
It was later realized that this string may legally exist in XML attributes and comments, why a later version of the spec defines a totally different framing mechanism based on length encoded chunks, much like the HTTP chunking mechanism.
If you have control of the clients and what they produce, a simple delimiter based approach like the original NETCONF solution should work fine.
I saw this description in the Oracle website:
"Since TCP by its nature is a stream based protocol, in order to reuse an existing connection, the HTTP protocol has to have a way to indicate the end of the previous response and the beginning of the next one. Thus, it is required that all messages on the connection MUST have a self-defined message length (i.e., one not defined by closure of the connection). Self demarcation is achieved by either setting the Content-Length header, or in the case of chunked transfer encoded entity body, each chunk starts with a size, and the response body ends with a special last chunk."
See Oracle doc
I don't know how to implement, can someone give me an example of Java implementation ?
If you are trying to implement "self-demarcation" in the same way as HTTP does it:
the HTTP 1.1 specification defines how it works,
the source code of (say) the Apache HTTP libraries are an example of its implementation.
In fact, it is advisable NOT to try and implement this (HTTP) yourself from scratch. Use an existing implementation.
On the other hand, if you simply want to implement your own ad-hoc self-demarcation scheme, it is really easy to do.
The sender figures out the size of the message, in bytes or characters or some other unit that makes sense.
The sender sends a the message size, followed by the message itself.
At the other end:
The receiver reads the message size, and then reads the requisite number of bytes, characters, to form the message body.
An alternative is to for the sender to send the message followed by a special end-of-message marker. To make this work, either you need to guarantee that no message will contain the end-of-message marker, or you need to use some sort of escaping mechanism.
Implementing these schemes is simple Java programming.
What makes a connection reusable
That is answered by the text that you quoted in your Question.
Entire code is quire complicated so I am directly coming to the point.
Code is as follows
SSLContext ctx = SSLContext.getInstance("TLS");
If you read docs for getInstance(String protocol) method it says
This method traverses the list of registered security Providers, starting
with the most preferred Provider. A new SSLContext object encapsulating
the SSLContextSpi implementation from the first Provider that supports the
specified protocol is returned.
Note that the list of registered providers may be retrieved via the
Security.getProviders() method.
For me Security.getProviders() method gives following providers
Now I have verified that "TLS" protocol is in com.sun.net.ssl.internal.ssl.Provider (index 2 ) and is always selected.
But the corresponding SSLContextSpi object is coming different in Java 6 and Java 7. In java 6 I am getting com.sun.net.ssl.internal.ssl.SSLContextImpl#7bbf68a9 and in java 7 I am getting sun.security.ssl.SSLContextImpl$TLS10Context#615ece16. This is having very bad effect as when later I am creating SSL socket they are of different class.
So why is this happening? Is there a work around? I want the same com.sun.net.ssl.internal.ssl.SSLContextImpl#7bbf68a9 SSLContextSpi object encapsulated in com.sun.net.ssl.internal.ssl.Provider context(which is same in both cases).
This is having very bad effect as when later I am creating SSL socket they are of different class.
This is not a bad effect. Which actual class you get from the factories in the public API is at the discretion of the JRE implementation: these concrete classes are not part of the public API.
The fact that you get different classes between Java 6 and Java 7 doesn't really matter. Even if they had the same name, if wouldn't make sense to compare them to one another.
EDIT:
public int read(byte[] b) function reads only 1 bytes when I give it a
byte array of length 4 and also i have confirmed that there are 4
bytes in the stream.
SSLSocket in Java 7 is behaving correctly when you get this. In fact, it's probably behaving better, since this initial 1-byte read is due to the BEAST-prevention measure. I'll copy and paste my own answer to that question, since you're making exactly the same mistake.
The assumption you're making about reading the byte[] exactly as you write them on the other end is a classic TCP mistake. It's not actually specific to SSL/TLS, but could also happen with a TCP connection.
There is no guarantee in TCP (and in SSL/TLS) that the reader's buffer will be filled with the exact same packet length as the packets in the writer's buffer. All TCP guarantees is in-order delivery, so you'll eventually get all your data, but you have to treat it as a stream.
This is why protocols that use TCP rely on indicators and delimiters to tell the other end when to stop reading certain messages.
For example, HTTP 1.1 uses a blank line to indicate when the headers end, and it uses the Content-Length header to tell the recipient what entity length to expect (or chunked transfer encoding). SMTP also uses line returns and . at the end of a message.
If you're designing your own protocol, you need to define a way for the recipient to know when what you define as meaningful units of data are delimited. When you read the data, read such indicators, and fill in your read buffer until you get the amount of bytes you expect or until you find the delimiter that you've defined.
I'm writing an application server that will receive SIP and DNS messages from the network.
When I receive a message from the network, I understand from the documentation that at first, I get a ChannelBuffer. I would like to determine which kind of message has been received (SIP or DNS) and to decode it.
To determine the message type, I can dedicate port to each type of message, but I would be interested to know if there exist another solution for that. My question is more about how to decode the ChannelBuffer.
Is there a ChannelHandler provided by Netty to decode SIP or DNS messages? If not, what would be the right place in the type hierarchy to write my custom ChannelHandler?
To illustrate my question, let's take as example the HttpRequestDecoder, the hierarchy is:
java.lang.Object
org.jboss.netty.channel.SimpleChannelUpstreamHandler
org.jboss.netty.handler.codec.frame.FrameDecoder
org.jboss.netty.handler.codec.replay.ReplayingDecoder<HttpMessageDecoder.State>
org.jboss.netty.handler.codec.http.HttpMessageDecoder
org.jboss.netty.handler.codec.http.HttpRequestDecoder
Also, do I need to use two different ChannelHandler for decoding and encoding, or is there a possibility to use a single ChannelHandler for both?
Thanks
If you really have a requirement for port unification (an example here), i.e. receiving different protocols on the same port, then you would have to detect the protocol in a handler and take appropriate actions. Could be as simple as inserting different handlers in the pipe line.
However, I find it very improbable that SIP and DNS would share the same port, hence no need to complicate matters.
I haven't seen a SIP decoder/encoder for Netty, but depending on what you want to do with the message, the HTTP decoder is a a very good starting point (and could be made simpler since chunking is not supported in SIP).
I would strongly recommend not to try to combine DNS and SIP decoding in one handler (or any other combination for that matter). Keep the handlers as simple and coherent as possible. Combine handlers instead, if needed.
This is the situation:
I'm working on a project where I need to be able to send one or more images once in a while to/from the server as well as a lot of other types of data represented with text. The way it is currently done, is by sending a message with that says "incoming image of size x to be used as y" (It's not "formulated" that way of course), and then I call a method that reads the next x bytes through a DataInputStream. At first I met some problems with latency screwing things up, but I made the server spawn a new thread to send the "incoming image" message, and then wait for a flag that is set when the client responds with a "I'm ready for the image" message. It works in a way now, but if anything else, for instance a chat message, is sent while the image is being transfered, that message meant for a BufferedReader will be read as raw bytes and used as part of the image. So I will have to block all outgoing data (and add it to a queue) when there is an image that is being sent. But this seems very wrong and annoying, as users of the application will not be able to chat while receiving/ uploading a big image.
This is what I need:
So, I either need to set up an independent channel to use for raw data. Which, as far as I understand from some tinkering, I will have to set up a new socket over a new port, which seems unnecessary. The other way I can see to solve this, would be to somehow use tag each packet with a "this is text/raw data" bit, but I have no idea how to do this with java? Can you add information to the packet-header when you write something to the stream (that every packet containing that info will contain) and then read the packet data on the other end and act accordingly?
As you can see, I do not have much experienced with networking, nor have I used Java for a long time. This is also my first post here, so be kind. If anything was unclear, please ask, and I'll specify. All ideas are welcome! (There is possibly a standard way to do this?)
Thanks a lot!
There is nothing in TCP protocol itself that can help you.
You either open a new socket connection (can be to the same server port), or you split your images in, smaller chunks and wrap these chunks in envelopes saying what type of message it is: image or chat. And then reconstruct the image on the receiving end from these chunks. But this will waste bandwidth and add complexities of its own (e.g. how big do you make a chunk of that image?).
I would go with the separate binary data connection.
Java should have a standard support for HTTP protocol - use HTTP to do your picture transfers as you can set the type of data being transmitted in the header. Basically, you would have your client/server architecture establish a separate request for each new data transfer (be it text or image), that way enabling you to do processing in a simple loop.
This might be of some help to you : How to use java.net.URLConnection to fire and handle HTTP requests?