I've written Socket Communication server with Java and a AIR programm with AS3, using Socket connection.
The communication through socket connection is done with JSON serialization.
Sometimes with really long JSON strungs over socket, AS3 code says that there is a JSON parse error.
Each JSON string I end with end string to let programm know, that it is not the end of the message, so this is not the problem with AIR programm reading the message in parts.
The error occurs only with realy long json string, for example, string with 78031 length. Is there any limits for JSON serialization?
I had the same problem. The problem is in Flash app reading data from socket.
The point is that Flash ProgressEvent.SOCKET_DATA event fires even when server didn't send all the data, and something is left (especially when the data is big and the connection is slow).
So something like {"key":"value"} comes in two (or more) parts, like: {"key":"val and ue"}. Also sometimes you might receive several joined JSONs in one message like {"json1key":"value"}{"json2key":"value"} - built-in Flash JSON parser cannot handle these too.
To fight this I recommend you to modify your SocketData handler in the Flash app to add a cache for received strings. Like this:
// declaring vars
private var _socket:Socket;
private var _cache: String = "";
// adding EventListener
_socket.addEventListener(ProgressEvent.SOCKET_DATA, onSocketData);
private function onSocketData(e: Event):void
{
// take the incoming data from socket
var fromServer: ByteArray = new ByteArray;
while (_socket.bytesAvailable)
{
_socket.readBytes(fromServer);
}
var receivedToString: String = fromServer.toString();
_cache += receivedToString;
if (receivedToString.length == 0) return; // nothing to parse
// convert that long string to the Vector of JSONs
// here is very small and not fail-safe alghoritm of detecting separate JSONs in one long String
var jsonPart: String = "";
var jsonVector: Vector.<String> = new Vector.<String>;
var bracketsCount: int = 0;
var endOfLastJson: int = 0;
for (var i: int = 0; i < _cache.length; i++)
{
if (_cache.charAt(i) == "{") bracketsCount += 1;
if (bracketsCount > 0) jsonPart = jsonPart.concat(_cache.charAt(i));
if (_cache.charAt(i) == "}")
{
bracketsCount -= 1;
if (bracketsCount == 0)
{
jsonVector.push(jsonPart);
jsonPart = "";
endOfLastJson = i;
}
}
}
// removing part that isn't needed anymore
if (jsonVector.length > 0)
{
_cache = _cache.substr(endOfLastJson + 1);
}
for each (var part: String in jsonVector)
{
trace("RECEIVED: " + part); // voila! here is the full received JSON
}
}
According to Adobe, it appears that you are not facing a JSON problem but instead a Socket limitation.
A String you may send over a Socket via writeUTF and readUTF is limited by 65,535 bytes. This is due to the string being prepended with a 16 bit unsigned integer rather than a null terminated string.
Related
I aim to use code via https://github.com/davidgyoung/ble-advert-counter/blob/master/app/src/main/java/com/radiusnetworks/blepacketcounter/MainActivity.java
to scan and read BLE device's adverting data.
The code works well. I could get the formatted adverting data via LogCat as pic shown.
But in the code I can't find the related log statement.
I didnt see BluetoothLeScanner class or onScanResult() method invoked.
And I want to obtain the String "ScanResult{mDevice=F3:E5:7F:73:4F:81, mScanRecord=ScanRecord..." to get the formatted data value.
How can I achieve this?
Thanks
I'm not sure about the logs but here's how you can get the data.
onLeScan() callback has all the information that is being printed in the logs. To get the device information you can use the device object from the call back(ex. device.getAddress()). Scan record will be in the callback's scanRecord byte array. You need to parse the array to get the information. I've used below code to parse the scan information.
public WeakHashMap<Integer, String> ParseRecord(byte[] scanRecord) {
WeakHashMap<Integer, String> ret = new WeakHashMap<>();
int index = 0;
while (index < scanRecord.length) {
int length = scanRecord[index++];
//Zero value indicates that we are done with the record now
if (length == 0) break;
int type = scanRecord[index];
//if the type is zero, then we are pass the significant section of the data,
// and we are thud done
if (type == 0) break;
byte[] data = Arrays.copyOfRange(scanRecord, index + 1, index + length);
if (data != null && data.length > 0) {
StringBuilder hex = new StringBuilder(data.length * 2);
// the data appears to be there backwards
for (int bb = data.length - 1; bb >= 0; bb--) {
hex.append(String.format("%02X", data[bb]));
}
ret.put(type, hex.toString());
}
index += length;
}
return ret;
}
Refer the below link to understand about the ble date advertisement.
BLE obtain uuid encoded in advertising packet
Hope this helps.
I have a code where i read a large file in chunks and transform it into JSON to send it to a Spring MVC controller, my method looks something like this, i pass where i want to start in the offset and the chunkSize in the noBytes, the result is passed to be transformed into JSON and sent to my controller
public String readByteBlock(File file, int offset, int noBytes) throws IOException {
InputStream in = new FileInputStream(file);
byte[] result = new byte[noBytes];
in.skip(offset);
in.read(result, 0, noBytes);
return FileSaver.byteArrayToString(result, noBytes);
}
I am having problem to make the next and previous pagination work in my controller
#RequestMapping("/page/{id}")
#ResponseBody
File file = new File("C:/teste.txt");
FileSaver fs = new FileSaver();
int byteblock = 100;
int offset = 0;
if(id.equals("next")){
offset = (int) req.getSession().getAttribute("prevEndPos");
String s = fs.readByteBlock(file, offset, byteblock);
req.getSession().setAttribute("prevStartPos", offset);
req.getSession().setAttribute("prevEndPos", (offset + byteblock) );
return s;
}
if(id.equals("prev")){
offset = (int) req.getSession().getAttribute("prevStartPos") - byteblock ;
if(offset < 0 ){
String s = fs.readByteBlock(file, 0, byteblock);
return s;
}
String s = fs.readByteBlock(file, offset, byteblock);
req.getSession().setAttribute("prevStartPos", (offset - byteblock) );
return s;
}
String s = fs.readByteBlock(file, 0, byteblock);
req.getSession().setAttribute("prevStartPos", offset);
req.getSession().setAttribute("prevEndPos", (offset + byteblock));
return s;
So each chunk of the file will be put into an html table in my application,i saved the current block startByte and endByte into my session but when the user press the previous button on my page how do i know where the previous block started and ended? since the session will hold the current page and not the previous one
First a little general about pagination.
Simple pagination only has two properties:
A page size
A page number
The back-end will sends back the data and the following metadata:
Total number of pages
The current page number
The current page size
The client is then responsible for asking for the chunk of data it wants, by sending the page number and page size, the backend will then calculate the offset and end. The only thing to be aware of is that if the user increases the page size, then the total number of pages (that the client received in the previous call) is no longer valid, but in those case the server just returns no data, and new metadata.
This approach is used by many frameworks, like Spring Data on the backend, and EXT JS for front-end.
Since you are using Spring, you should just create a DTO class that holds both the data and meta data:
Class PageResult {
String data;
int totalPageCount;
int currentPage;
int pageSize;
}
In you are using String for the transporting data be aware that you may have to call JSON.parse() on the front-end, and your JSON will be string escaped which looks silly; if you are reading you data using Jacksons ObjectMapper I would recommend using a ObjectMapper.readTree() which returns a JsonNode, and put that into your DTO class.
On the front-end you can do a dropdown with suggested page sizes, and an input box for the page number. If you want next and previous, just send a request with current page +/- 1 and the page size, then the backend will calculate start, end and total pages.
Instead of adding two int attributes to your session, why not add two arrays of ints, containing the appropriate values? And also add a 'page' attribute, so you can do something like
if(id.equals("prev") {
List<Integer> offsetStarts = req.getSession.getAttribute("offsetStarts);
List<Integer> offsetEnds = req.getSession.getAttribute("offsetEnds);
int lastPage = req.getSession.getAttribute("page");
int desiredPage = lastPage -1;
int offsetStart = offsetStarts.get(desiredPage);
int offsetEnd = offsetEnd.get(desiredPage);
// get the chunk as desired, and return it
req.getSession().setAttribute("page", desiredPage);
}
// if it's a new chunk, append offsets to the arrays
I'm currently developing a system that gets data from a battery pack of an electric vehicle, stores it in a database and display it on a screen.
So I have a Java - Application that reads the data from a hardware interface, interprets the values and sends it via Socket to a Node.js-Server. (Java App and Webserver are running on the same computer, so Url = localhost)
JAVA APP:
s = new Socket();
s.connect(new InetSocketAddress(URL, PORT));
out = new PrintWriter( s.getOutputStream(), true);
for (DataEntry e : entries){
out.printf(e.toJson());
}
NODE:
sock.on('data', function(data) {
try{
var data = JSON.parse(data);
db.serialize(function(){
db.run("INSERT INTO DataEntry(value, MessageField, time) values(" + data.value + "," + data.messageFieldID + ", STRFTIME('%Y-%m-%d %H:%M:%f'))");
});
} catch(e){}
});
I get about 20 Messages per second from the hardware interface which are converted into 100 Json - Strings. So the webserver has to process one message in 10 ms, which I thought, is manageable.
But here is the problem: If my entries - List (foreach loop) has more than 2 elements, the webserver gets 2 or more of the Json's in one message.
So the first message was divided into 2 parts (ID 41,42) and was processed correctly. But the second message was divided into 5 parts (ID 43-47), and the first 4 of them weren't sent alone, so only the last one was saved correctly.
How can I ensure, that every Json is sent one another?
Isn't there something like a buffer so that the socket.on method is called correctly for every message I send?
I hope somebody of you can help me
Thank you!
Benedikt :)
TCP sockets are just streams and you shouldn't make any assumptions about how much of a "message" is contained in a single packet.
A simple solution to this is to terminate each message with a newline character since JSON cannot contain such a character. From there it's a simple matter of buffering data until you see a newline character. Then call JSON.parse() on your buffer. For example:
var buf = '';
sock.on('data', function(data) {
buf += data;
var p;
// Use a while loop as it may be possible to have multiple
// messages buffered depending on chunk contents
while (~(p = buf.indexOf('\n'))) {
try {
var msg = JSON.parse(buf.slice(0, p));
} catch (ex) {
console.log('Bad JSON message: ' + ex);
}
buf = buf.slice(p + 1);
}
});
You will also need to change printf() to println() on the Java-side so that a newline character will be appended to each message.
I am going to use Twitter for some semantic text analysis in a school class. I downloaded the Hosebird Client for Java and is running the FilterStreamExample.java: https://github.com/twitter/hbc/blob/master/hbc-example/src/main/java/com/twitter/hbc/example/FilterStreamExample.java
Running it, I get a lot of data about the users' profiles, their settings, background images, etc. I just want the tweeted text only. And maybe the location and username.
It may be a stupid question, but how do I make it only display the "texts" information? Right now, it just prints out everything.
// Do whatever needs to be done with messages
for (int msgRead = 0; msgRead < 1000; msgRead++) {
String msg = queue.take();
System.out.println(msg);
}
I could probably do a search for "text" in the strings themselves, but it seems a bit cumbersome. Isn't there any better way to do it?
The response from the twitter Streaming API is JSON String. Parse the string into JSON Object and get the value from the key "text"
import org.json.*;
for (int msgRead = 0; msgRead < 1000; msgRead++) {
String msg = queue.take();
JSONObject obj = new JSONObject(msg);
String text= obj.getString("text");
System.out.println(msg);
}
*Not Tested
Refer the following for parsing JSON in Java
How to parse JSON in Java
I'm currently developing a client/server architecture between a tablet (client) and a MAC/PC (server). I am doing on both side some real-time rendering and I need communication between the two.
The problem is that I need to do some operation on the string I get from my client (which is basically a rotation matrix). This string is therefore to be at most 16 float numbers that I previously transform into a coma-separated-value string.
Therefore what I should get from my client is something like:
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
Server-side, I do some processing of that string to get back my rotation matrix as a float array of 16 elements. The problem is that sometimes I get more than just 16 elements from the client on the server side at once. I for instance get
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
So that when I try to split it, I go above the 16 element limits which is not good at all for me.
My question is: is there a way to prevent the server and/or the client to read/send more than one complete matrix at a time? Since I'm using a tablet and some real-time rendering I would like to be able to save as much processing power as possible.
Here is the code that I'm using (just snippets as files are quite big)
Client:
if (connected == true && matrixupdated == true && this.hasMatrixChanged()){
try {
this.inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
this.outToServer= new DataOutputStream(clientSocket.getOutputStream());
this.sentence = this.getStringFromMatrix();
outToServer.writeBytes(sentence + '\n');
this.hasServerProcessed = false ;
System.arraycopy(matrix, 0, previousMatrix, 0, 16); //I check whether the matrix changes enough for me to send it to the server
}catch (Exception e) {
Log.e("ClientActivity", "S: Error", e);
}
this.matrixupdated = false ;
Server :
while( (read_size = recv(sock , client_message , 2000 , 0)) > 0 )
{
smatrix = client_message ; //smatrix is a true c++ string
pthread_mutex_lock(&mymutex);
pthread_cond_wait(&mycondition, &mymutex); // prevent real-time rendering to try and use the matrix at the same time as this function
std::stringstream ss(smatrix);
while(std::getline(ss, tok, ',')) {
matrix[i] = ::atof(tok.c_str());
i++ ;
}
i = 0 ;
pthread_mutex_unlock(&mymutex);
}
Working as designed. TCP is a byte stream protocol. There are no message boundaries. If you want messages you have to implement them yourself.