Develop Reference

Iterating Text in hadoop

I am trying to iterate through a Text and print its contents. This is the code I tried:
Text text = new Text();
text.set("Hadoop");
ByteBuffer buf = ByteBuffer.wrap(text.getBytes(),0,text.getLength());
int cp = 0;
while(buf.hasRemaining() && (cp = Text.bytesToCodePoint(buf)) != 1)
System.out.println(Integer.toHexString(cp));
This is printing me the code points. How to print the characters from this?
EDIT
For the input "Hadoop", casting the int cp to a char inside your while loop works. BUT, when the text is something like \u0041\u00DF\u6771\uD801\uDC00 then using the same code I am getting a couple of ? printed in the console. Any specific reasons for this? Please suggest.

I guess the easiest way would be for you to just cast your ints to chars. Like so:
int[] chars = { 0x41, 0xdf, 0x6671, 0x10400 };
for(int c : chars) {
String out = String.format("%d -> %s", c, (char) c);
System.out.println(out);
}
My output is:
65 -> A
223 -> ß
26225 -> 晱
66560 -> Ѐ

Extract Bytes At specified location from byte array

hello i am new with byte manipulation in java. i already have byte array with flowing format
1-> datapacketlength (length of name) (first byte)
2-> name (second byte + datapacket length)
3-> datapacketlength (length of datetime)
4-> current date and time
how can i extract the name and current date and time.should i use Arrays.copyOfRange() method.
Regards from
mcd

You can use ByteBuffer and use your current byte array, then use the methods that come with it to get the next float, int etc (such as buffer.getInt and buffer.getFloat).
You can get a portion of your byte array when you create a new bytebuffer by using the wrap method I believe. The possibilities are endless :). To get strings as you asked, you simply need to do something like:
byte[] name = new byte[nameLength];
buffer.get(name);
nameString = byteRangeToString(name);
where byteRangeToString is a method to return a new string representation of the byte[] data you pass it.
public String byteRangeToString(byte[] data)
{
try
{
return new String(data, "UTF-8");
}
catch (UnsupportedEncodingException e)
{
/* handle accordingly */
}
}
See: http://developer.android.com/reference/java/nio/ByteBuffer.html
Using copyOfRange() may run you into memory issues if used excessively.

What about something like :
int nameLength = 0;
int dateLength = 0;
byte[] nameByteArray;
byte[] dateByteArray
for(int i=0; i<bytesArray.length; i++){
if(i == 0){
nameLength = bytesArray[i] & 0xFF;
nameByteArray = new byte[nameLength];
}
else if(i == nameLength+1){
dateLength = byteArray[i] & 0xFF;
dateByteArray = new byte[dateLength];
}
else if(i < nameLength+1){
nameByteArray[i-1] = bytesArray[i];
}
else{
dateByteArray[i-(nameLength+1)] = bytesArray[i];
}
}

You want to use a DataInputStream.

what is the best way override image parsing on com.google.Volley?

I developed my own rest api using c# servicestack on mono. It works as expected except when it comes to file download. I noticed it appends some bits at the start of file. for example see the image below:
I filed bug to mono bugzilla, Meanwhile, I want to override image response on my client to remove first appended stuff to make image work. I tried it on c# client by editing received stream before saving it to file and it works fine.
I need to know how and where I can override volley library to get clean images not malformed ones with the best performance.
Update 04:37 PM:
I believe I need to modify com.android.volley.toolbox.ImageRequest >> I will try it and post solution if it works with me.
Regards,
Shaheen

I modified the method doParse in com.android.volley.toolbox.ImageRequest
private Response<Bitmap> doParse(NetworkResponse response){
byte[] data = response.data;
byte [] pattern = fromHexString("FFD8FFE000");
int position = matchPosition(data,pattern);
if(position>0)
data = Arrays.copyOfRange(data, position, data.length-position);
....
....
....
....
..}
here is the helper methods I used:
public static int matchPosition(byte [] a, byte [] b){
int matchLength=0;
int maxSearch = 30>a.length?a.length:30;
for (int i =0;i<maxSearch;i++) {
if (a[i]==b[0] && i+b.length<a.length){
for(int j = 0;j< b.length;j++ )
{
if((i+j)==a.length-1)
return -1;
if(a[i+j]==b[j])
matchLength++;
}
if(matchLength == b.length)
return i;
else
matchLength = 0;
}
}
return -1;
}
private static byte[] fromHexString(final String encoded) {
if ((encoded.length() % 2) != 0)
throw new IllegalArgumentException("Input string must contain an even number of characters");
final byte result[] = new byte[encoded.length()/2];
final char enc[] = encoded.toCharArray();
for (int i = 0; i < enc.length; i += 2) {
StringBuilder curr = new StringBuilder(2);
curr.append(enc[i]).append(enc[i + 1]);
result[i/2] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
and this work around resolved the issue explained above!

Sending messages to java server in objective-c via AsyncSocket

Hi I am trying to send messages to a Java server that I cannot change from an iOS device. I am using AsyncSocket and was wondering how to send and receive the length appended to the string. I am doing a UTF conversion of the string to NSData however I was wondering if there is a difference in the size of the primitive between the two languages. Also is there a big endian and little endian variation? Basically I need to be able to convert the following java methods:
inStream.readUTF();
inStream.readInt();
inStream.readChar();
inStream.readShort();
inStream.readFully(recvBuff, 0, recvLen);
outStream.writeInt();
outStream.writeUTF();
outStream.writeChars();
outStream.writeShort();
outStream.write(sendBytes, 0, sendBytes.length);
I know I am very close but something is not quite right, this is what I have got so far:
I am using an NSMutableArray to append the data and the using AsyncSockets read and write methods.
[theSocket readDataToData:[AsyncSocket ZeroData] withTimeout:timeout buffer:buffer bufferOffset:offset tag:tag]; // inStream.readUTF();
[theSocket readDataToLength:sizeof(int32_t) withTimeout:timeout buffer:buffer bufferOffset:offset tag:tag]; // inStream.readInt();
[theSocket readDataToLength:sizeof(unichar) withTimeout:timeout buffer:buffer bufferOffset:offset tag:tag]; // inStream.readChar();
[theSocket readDataToLength:sizeof(int16_t) withTimeout:timeout tag:tag]; // inStream.readShort();
[theSocket readDataWithTimeout:timeout buffer:buffer bufferOffset:offset maxLength:maxLength tag:tag]; // inStream.readFully(recvBuff, 0, recvLen);
[outputBufferStream appendBytes:&[sendString length] length:sizeof([sendString length])]; // outStream.writeInt();
[outputBufferStream appendData:[sendString dataUsingEncoding:NSUTF8StringEncoding]] // outStream.writeUTF();
char array[5];
[outputBufferStream appendBytes:array length:sizeof(array)]; // outStream.writeChars();
int16_t _short;
[outputBufferStream appendBytes:&_short length:sizeof(_short)]; // outStream.writeShort();
unsigned char *sendBytes;
[outputBufferStream appendBytes:sendBytes length:sendBytesLength]; // outStream.write(sendBytes, 0, sendBytes.length);
I usually append the length at the beginning like so:
int32_t sendStringLength = [sendString length];
[outputBufferStream appendBytes:&sendStringLength length:sizeof(sendStringLength)];
At the end of the write i am appending the following as a terminator:
[outputBufferStream appendData:[#"\n" dataUsingEncoding:NSUTF8StringEncoding]];
I would really appreciate any help with this. Thanks.
EDIT::
I have got most of it working thanks to Robadob. Here is a little java snippet (working) of the bit i am currently stuck trying to get working on Objective-C:
private int sendData(String stringToSend) {
if (theSocket==null) {
lastError="sendData() called before socket was set up.";
return 1; // Error
}
try {
System.out.println("Sending "+stringToSend.length()+" chars ["+ stringToSend.length()*2+" bytes]");
System.out.println("'" + stringToSend + "'");
outStream.writeInt(stringToSend.length()*2);
outStream.writeChars(stringToSend);
outStream.flush();
} catch (IOException e) {
lastError="sendData() exception: "+e;
System.out.println(lastError);
return 2; // Error
}
return 0; // Ok
}
Here is a snippet of what I have got so far in Objective-C:
- (int32_t)sendData:(NSString *)stringToSend {
if (theSocket == nil) {
lastError = #"sendData called before socket was set up";
return 1; // Error
}
#try {
NSLog(#"Sending %d chars [%d bytes]", [stringToSend length], ([stringToSend length] * 2));
NSLog(#"'%#'", stringToSend);
uint32_t stringToSendInt = ([stringToSend length] * 2);
uint32_t stringToSendIntBigE = CFSwapInt32HostToBig(stringToSendInt);
[outputBufferStream appendBytes:&stringToSendIntBigE length:sizeof(stringToSendIntBigE)];
stringToSend = [stringToSend stringByAppendingString:#"\n"];
for (int i = 0; i < ([stringToSend length]); i++) {
unichar characterTmp = [stringToSend characterAtIndex:i];
unichar character = characterTmp << 8;
[outputBufferStream appendBytes:&character length:sizeof(character)];
}
[self syncWriteData:outputBufferStream withTimeout:socketTimeout tag:kSendDataSocketTag];
outputBufferStream = [NSMutableData data];
}
#catch (NSException *e) {
lastError = [NSString stringWithFormat:#"sendData exception: %#", [e reason]];
NSLog(#"%#", lastError);
return 2; // Error
}
return 0; // Ok
}

If you read the docs for writeUTF it says it writes the first 2 bytes using writeShort. This says
Writes a short to the underlying output stream as two bytes, high byte
first. If no exception is thrown, the counter written is incremented
by 2.
A byte is 8 bits, so that makes the value they are writing is 16 bits, you are using int32_t, which is 32 bits. you should be writing an int16 or int16_t (I don't objective-c).

Generating a .ov2 file with Java

I am trying to figure out how to create a .ov2 file to add POI data to a TomTom GPS device. The format of the data needs to be as follow:
An OV2 file consists of POI records. Each record has the following data format.
1 BYTE, char, POI status ('0' or '2')
4 BYTES, long, denotes length of the POI record.
4 BYTES, long, longitude * 100000
4 BYTES, long, latitude * 100000
x BYTES, string, label for POI, x =3D=3D total length =96 (1 + 3 * 4)
Terminating null byte.
I found the following PHP code that is supposed to take a .csv file, go through it line by line, split each record and then write it into a new file in the proper format. I was hoping someone would be able to help me translate this to Java. I really only need the line I marked with the '--->' arrow. I do not know PHP at all, but everything other than that one line is basic enough that I can look at it and translate it, but I do not know what the PHP functions are doing on that one line. Even if someone could explain it well enough then maybe I could figure it out in Java. If you can translate it directly, please do, but even an explanation would be helpful. Thanks.
<?php
$csv = file("File.csv");
$nbcsv = count($csv);
$file = "POI.ov2";
$fp = fopen($file, "w");
for ($i = 0; $i < $nbcsv; $i++) {
$table = split(",", chop($csv[$i]));
$lon = $table[0];
$lat = $table[1];
$des = $table[2];
--->$TT = chr(0x02).pack("V",strlen($des)+14).pack("V",round($lon*100000)).pack("V",round($lat*100000)).$des.chr(0x00);
#fwrite($fp, "$TT");
}
fclose($fp);
?>

Load a file into an array, where each element is a line from the file.
$csv = file("File.csv");
Count the number of elements in the array.
$nbcsv = count($csv);
Open output file for writing.
$file = "POI.ov2";
$fp = fopen($file, "w");
While $i < number of array items, $i++
for ($i = 0; $i < $nbcsv; $i++) {
Right trim the line (remove all whitespace), and split the string by ','. $table is an array of values from the csv line.
$table = split(",", chop($csv[$i]));
Assign component parts of the table to their own variables by numeric index.
$lon = $table[0];
$lat = $table[1];
$des = $table[2];
The tricky bit.
chr(02) is literally character code number 2.
pack is a binary processing function. It takes a format and some data.
V = unsigned long (always 32 bit, little endian byte order).
I'm sure you can work out the maths bits, but you need to convert them into little endian order 32 bit values.
. is a string concat operator.
Finally it is terminated with chr(0). Null char.
$TT = chr(0x02).
pack("V",strlen($des)+14).
pack("V",round($lon*100000)).
pack("V",round($lat*100000)).
$des.chr(0x00);
Write it out and close the file.
#fwrite($fp, "$TT");
}
fclose($fp);

The key in JAVA is to apply proper byte order ByteOrder.LITTLE_ENDIAN to the ByteBuffer.
The whole function:
private static boolean getWaypoints(ArrayList<Waypoint> geopoints, File f)
{
try{
FileOutputStream fs = new FileOutputStream(f);
for (int i=0;i<geopoints.size();i++)
{
fs.write((byte)0x02);
String desc = geopoints.get(i).getName();
int poiLength = desc.toString().length()+14;
fs.write(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(poiLength).array());
int lon = (int)Math.round((geopoints.get(i).getLongitudeE6()/1E6)*100000);
fs.write(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(lon).array());
int lat = (int)Math.round((geopoints.get(i).getLatitudeE6()/1E6)*100000);
fs.write(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(lat).array());
fs.write(desc.toString().getBytes());
fs.write((byte)0x00);
}
fs.close();
return true;
}
catch (Exception e)
{
return false;
}
}