I have to build a serial comunication app using python, the old app is only working on windows xp and was written in C. Now i have to switch to linux and i dont have a working driver. I started to code it myself. I got the protocol from the producing company of the serial device.
The serial device receives and sends data wich is ended by a CRC. I'm new to python and i dont have a solution for this, maybe someone can help me.
This is the CRC alghoritm:
ALGORITHM FOR CRC CALCULATION
The two CRC bytes are calculated according to the formula x^15 + 1. In the calculation are included all data bytes plus the byte for block end. Every byte passes through the calculation register from teh MSB to LSB.
Three working bytes are used - S1, S0 and TR
S1 - Most significant byte from the CRC ( it is transmitted immediatelly after END)
S0 - Least significant byte from the CRC ( It is transmitted after S1)
TR - the current transmitted byte in the block.
The CRC is calculated as follows:
1. S1 and S0 are zeroed
2. TR is loaded with the current transmitted byte. The byte is transmitted.
3. Points 3.1 and 3.2 are executed 8 times:
3.1. S1, S0 and TR are shifted one bit to the left.
3.2. If the carry bit from S1 is 1, the MSB of S1 and LSB of S0 are inverted.
Points 2 and 3 are executed for all bytes, included in the calculation of the CRC - from the first byte after BEG up to and including byte END.
4. TR is loaded with 0 and point 3 is executed
5. TR is loaded with 0 and point 3 is executed
6. Byte S1 is transmitted
7. Byte S0 is transmitted
ALGORITHM FOR CRC CHECK ON RECEIVING
Three working bytes are used S1, S0 and RC
S1 - Most significant byte from the CRC ( it is received immediately after END)
S0 - Least significant byte from the CRC ( transmitted after S1)
RC - the current received byte in the block ( beginning from the first byte after BEG and ending 2 bytes after END).
The CRC is obtained as follows:
1. S1 and S0 are zeroed
2. RC is loaded with the current received byte
3. Points 3.1 and 3.2 are executed 8 times:
3.1. S1, S0 and RC are shifted 8 times to the left
3.2. if the MSB of S1 is 1 then MSB of S1 and LSB of S0 are inverted.
Points 2 and 3 are executed for all bytes, included in the calculation of the CRC - from the first byte after BEG up to and including 2 bytes after END.
S1 and S0 must be 0.
If there is someone wich can show me how to do it i'll be very gratefull.Thank you all.
EDIT 1:
I managed to get the same CRC procedure made by someone, but its made in java, i'm not that good with java. Maybe you can guide me into converting it in python. This is the code:
public class Crc {
public static final String CRC_NAME = "CRC-16-ECR";
private static final int POLYNOMIAL = 32769;
public static final int WIDTH = 16;
public static final int TOPBIT = 32768;
short CRC;
short[] crcTable = new short[256];
public Crc() {
this.crcInit();
}
private void crcInit() {
for(int dividend = 0; dividend < 256; ++dividend) {
int remainder = dividend << 8;
for(byte bit = 8; bit > 0; --bit) {
if((remainder & '耀') != 0) {
remainder = (remainder << 1 ^ '老') & '\uffff';
} else {
remainder = remainder << 1 & '\uffff';
}
}
this.crcTable[dividend] = (short)remainder;
}
}
public short crcFast(byte[] message, int nBytes) {
int remainder = 0;
for(int oneByte = 0; oneByte < nBytes; ++oneByte) {
int data = (message[oneByte] ^ remainder >> 8) & 255;
remainder = this.crcTable[data] ^ remainder << 8;
}
return (short)remainder;
}
}
#!/usr/bin/python
import sys
crc = 0
while True:
ch = sys.stdin.read(1)
if not ch:
break
crc ^= ord(ch) << 8
for _ in range(8):
crc = crc << 1 if (crc & 0x8000) == 0 else (crc << 1) ^ 0x8001
crc &= 0xffff
print(format(crc, '04x'))
Related
Given an array filled with 4 bytes inside (R,G,B,A), I'm trying to translate this array full of 4 8bits numbers into its translation in 32bits. To be more clear, if I get an array such as:
byte[] tab = {1,2,3,4};
with translated in binary in 8bit :
1 = 0b00000001
2 = 0b00000010
3 = 0b00000011
4 = 0b00000100
Then, my method should return a byte array such as :
newTab = {00000001_00000010_00000011_00000100};
For some reason, I'm trying to do this without using a String to concatenate the bytes.
I've already tried something with binary operators such as <<, >> or |, but without success...
So far, my code looks like this :
byte[] tab = {1,2,3,4};
int tmp,tabToInt = 0;
for (int x = 0 ; x < tab.length ; ++x){
tmp = tmp << (tab.length - 1 - x)*8;
byteToInt = byteToInt | tmp;
}
return tabToInt;
But it didn't seem to work, even less with negatives bytes... (like -1 = 0b11111111)
Thanks in advance for your answers!
You can use ByteBuffer like this.
byte[] tab = {1, 2, 3, 4};
int tabToInt = ByteBuffer.wrap(tab).getInt();
System.out.println("decimal = " + tabToInt);
System.out.println("binary = " + Integer.toBinaryString(tabToInt));
System.out.println("hexadecimal =" + Integer.toHexString(tabToInt));
output
decimal = 16909060
binary = 1000000100000001100000100
hexadecimal =1020304
ByteBuffer can do it, but only if you get passed at least 4 bytes.
The problem with your code is two-fold:
I think you typoed somewhere, your code doesn't even compile. I think you meant tmp = tab[x] << (tab.length - 1 - x)*8;. Your snippet never does anything with tab other than ask for its length.
Negative numbers extend, and java will convert any byte or short to an int the moment you do any math to it. So, 0b1111 1111, if you try to do e.g. << 8 on that, java first turns that -1 byte into a -1 int (so that's now 32 1 bits), and then dutifully left shifts it by 8, so now that's 24 1 bits, followed by 8 0 bits. You then bitwise OR that into your target, and thus now the target is mostly 1 bits. To convert a byte to an int without "sign extension", (b & 0xFF does it:
byte b = (byte) 0b1111_1111;
assert b == -1; // yup, it is
int c = b; // legal
assert c == -1; // yeah, still is. uhoh. That's...
int d = 0b11111111_11111111_11111111_11111111;
assert c == d; // yeah. We don't want that.
int e = (b & 0xFF);
assert e = 255;
int f = 0b0000000_0000000_0000000_11111111;
assert e == f; // yes!
What is the best method to create a hash of String, if the hash may not have more than 4 characters, and those 4 characters may only be lowercase letters or digits?
The strings I want to hash have 1-255 characters.
I know that it's probably impossible to create as 4-char hash without collision. But it would be sufficient if I'd have a good hash where possible collisions are minimized.
What I tried is the CRC16CCITT from here:
http://introcs.cs.princeton.edu/java/61data/CRC16CCITT.java
public class CRC16CCITT {
public static void main(String[] args) {
int crc = 0xFFFF; // initial value
int polynomial = 0x1021; // 0001 0000 0010 0001 (0, 5, 12)
// byte[] testBytes = "123456789".getBytes("ASCII");
byte[] bytes = args[0].getBytes();
for (byte b : bytes) {
for (int i = 0; i < 8; i++) {
boolean bit = ((b >> (7-i) & 1) == 1);
boolean c15 = ((crc >> 15 & 1) == 1);
crc <<= 1;
if (c15 ^ bit) crc ^= polynomial;
}
}
crc &= 0xffff;
StdOut.println("CRC16-CCITT = " + Integer.toHexString(crc));
}
}
But this gives too many collision. Are there better algorithms?
You are mistaking "hexadecimal digits" for "characters":
int crc = 0xFFFF; // initial value
That's only 2 bytes (0xFF is just 1 byte). For a CRC of 4 ANSI characters, you need 4 bytes (0xFFFFFFFF).
You'll have to adapt the rest of the code to work with double the legth, please comment if you don't know how to do that.
PS: You could do it with less than 4 bytes, but that would complicate things more than necessary.
I don't understand what is this doCalculatePi means or does, in the following example:
public static double doCalculatePi(final int sliceNr) {
final int from = sliceNr * 10;
final int to = from + 10;
final int c = (to << 1) + 1;
double acc = 0;
for (int a = 4 - ((from & 1) << 3), b = (from << 1) + 1; b < c; a = -a, b += 2) {
acc += ((double) a) / b;
}
return acc;
}
public static void main(String args[]){
System.out.println(doCalculatePi(1));
System.out.println(doCalculatePi(2));
System.out.println(doCalculatePi(3));
System.out.println(doCalculatePi(4));
System.out.println(doCalculatePi(10));
System.out.println(doCalculatePi(100));
}
I have printed the values to understand what the results are but I still have no clue what this code calculates. The conditions inside the loop are not clear.
<< means left shift operation, which shifts the left-hand operand left by the number of bits specified by the right-hand operand (See oracle docs).
Say, you have a decimal value, 5 which binary representation is 101
Now for simplicity, consider,
byte a = (byte)0x05;
Hence, the bit representation of a will be,
a = 00000101 // 1 byte is 8 bit
Now if you left shift a by 2, then a will be
a << 2
a = 00010100 //shifted place filled with zero/s
So, you may now understand that, left shift a by 3 means
a << 3
a = 00101000
For better understanding you need to study Bitwise operation.
Note, you are using int instead of byte, and by default, the int data type is a 32-bit signed integer (reference here), so you have to consider,
int a = 5;
in binary
a << 3
a = 00000000 00000000 00000000 00101000 // total 32 bit
My guess is that it approximates PI with
PI = doCalculatePi(0)+doCalculatePi(1)+doCalculatePi(2)+...
Just a guess.
Trying this
double d = 0;
for(int k = 0; k<1000; k++) {
System.out.println(d += doCalculatePi(k));
}
gives me
3.0418396189294032
3.09162380666784
3.1082685666989476
[...]
3.1414924531892394
3.14149255348994
3.1414926535900394
<< is the Bitshift operator.
Basically, every number is represented as a series of binary digits (0's and 1's), and you're shifting each of those digits to the left by however many places you indicate. So for example, 15 is 00001111 and 15 << 1 is 00011110 (or 30), while 15 << 2 is (00111100) which is 60.
There's some special handling that comes into play when you get to the sign bit, but you should get the point.
I have written a function that computes the checksum for a given tcp packet. However, when I capture a tcp packet sent over ipv4 from wireshark and let my function compute its checksum, then its not the same checksum as in the wireshark captured packet. I checked and the bytes I give to the computeChecksum function are exactly the same as the tcp packet bytes i captured with wireshark.
I computed the checksum according to the RFC 793. Does anybody see if there's anything wrong in my code?
public long computeChecksum( byte[] buf, int src, int dst ){
int length = buf.length; // nr of bytes of the tcppacket in total.
int pseudoHeaderLength = 12; // nr of bytes of pseudoheader.
int i = 0;
long sum = 0;
long data;
buf[16] = (byte)0x0; // set checksum to 0 bytes
buf[17] = (byte)0x0;
// create the pseudoheader as specified in the rfc.
ByteBuffer pseudoHeaderByteBuffer = ByteBuffer.allocate( 12 );
pseudoHeaderByteBuffer.putInt( src );
pseudoHeaderByteBuffer.putInt( dst );
pseudoHeaderByteBuffer.put( (byte)0x0 ); // store the 0x0 byte
pseudoHeaderByteBuffer.put( (byte)PROTO_NUM_TCP ); // stores the protocol number
pseudoHeaderByteBuffer.putShort( (short) length ); // store the length of the packet.
byte[] pbuf = pseudoHeaderByteBuffer.array();
// loop through all 16-bit words of the psuedo header
int bytesLeft = pseudoHeaderLength;
while( bytesLeft > 0 ){
// store the bytes at pbuf[i] and pbuf[i+1] in data.
data = ( ((pbuf[i] << 8) & 0xFF00) | ((pbuf[i + 1]) & 0x00FF));
sum += data;
// Check if the sum has bit 17 or higher set by doing a binary AND with the 46 most significant bits and 0xFFFFFFFFFF0000.
if( (sum & 0xFFFFFFFF0000) > 0 ){
sum = sum & 0xFFFF; // discard all but the 16 least significant bits.
sum += 1; // add 1 (because we have to do a one's complement sum where you add the carry bit to the sum).
}
i += 2; // point to the next two bytes.
bytesLeft -= 2;
}
// loop through all 16-bit words of the TCP packet (ie. until there's only 1 or 0 bytes left).
bytesLeft = length;
i=0;
while( bytesLeft > 1 ){ // note that with the pseudo-header we could never have an odd byte remaining.
// We do do exactly the same as with the pseudo-header but then for the TCP packet bytes.
data = ( ((buf[i] << 8) & 0xFF00) | ((buf[i + 1]) & 0x00FF));
sum += data;
if( (sum & 0xFFFF0000) > 0 ){
sum = sum & 0xFFFF;
sum += 1;
}
i += 2;
bytesLeft -= 2;
}
// If the data has an odd number of bytes, then after adding all 16 bit words we remain with 8 bits.
// In that case the missing 8 bits is considered to be all 0's.
if( bytesLeft > 0 ){ // ie. there are 8 bits of data remaining.
sum += (buf[i] << 8 & 0xFF00); // construct a 16 bit word holding buf[i] and 0x00 and add it to the sum.
if( (sum & 0xFFFF0000) > 0) {
sum = sum & 0xFFFF;
sum += 1;
}
}
sum = ~sum; // Flip all bits (ie. take the one's complement as stated by the rfc)
sum = sum & 0xFFFF; // keep only the 16 least significant bits.
return sum;
}
If you don't see anything wrong with the code then let me know that too. In that case I know to look somewhere else for the problem.
I've tested your code and it works correctly. I've done the following:
Configure wireshark to "Validate the TCP checksum if possible" in order to avoid to do the test with a packet with an incorrect checksum.
Add the long type suffix L to the constant 0xFFFFFFFF0000 in order to avoid the compile time error integer number too large (Java 8).
Use an hexadecimal representation of a TCP segment coming from wireshark
String tcpSegment = "0050dc6e5add5b4fa9bf9ad8a01243e0c67c0000020405b4010303000101080a00079999000d4e0e";
Use a method to convert an hexadecimal string to a byte array
public static byte[] toByteArray(String strPacket) {
int len = strPacket.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(strPacket.charAt(i), 16) << 4)
+ Character.digit(strPacket.charAt(i + 1), 16));
}
return data;
}
Use a ByteBuffer to write the source and destination adress into an int
int src = ByteBuffer.wrap(toByteArray("c0a80001")).getInt();
int dst = ByteBuffer.wrap(toByteArray("c0a8000a")).getInt();
With this, I obtain a checksum of C67C, the same as in wireshark.
P.S.: There is an error in your code when you do
pseudoHeaderByteBuffer.putShort( (short) length );
you store the length in two's-complement inside the pseudo header which will be a problem if the length is greater than 2^15. You better used char which is 16 bit unsigned.
I am trying to convert a HEX-sequence to a String encoded in either, ISO-8859-1, UTF-8 or UTF-16BE. That is, I have a String looking like: "0422043504410442" this represents the characters: "Test" in UTF-16BE.
The code I used to convert between the two formats was:
private static String hex2String(String hex, String encoding) throws UnsupportedEncodingException {
char[] hexArray = hex.toCharArray();
int length = hex.length() / 2;
byte[] rawData = new byte[length];
for(int i=0; i<length; i++){
int high = Character.digit(hexArray[i*2], 16);
int low = Character.digit(hexArray[i*2+1], 16);
int value = (high << 4) | low;
if( value > 127)
value -= 256;
rawData[i] = (byte) value;
}
return new String(rawData, encoding);
}
This seems to work fine for me, but I still have two questions regarding this:
Is there any simpler way (preferably without bit-handling) to do this conversion?
How am I to interpret the line: int value = (high << 4) | low;?
I am familiar with the basics of bit-handling, though not at all with the Java syntax. I believe the first part shift all bits to the left by 4 steps. Though the rest I don't understand and why it would be helpful in this certain situation.
I apologize for any confusion in my question, please let me know if I should clarify anything.
Thank you.
//Abeansits
Is there any simpler way (preferably without bit-handling) to do this conversion?
None I would know of - the only simplification seems to parse the whole byte at once rather than parsing digit by digit (e.g. using int value = Integer.parseInt(hex.substring(i * 2, i * 2 + 2), 16);)
public static byte[] hexToBytes(final String hex) {
final byte[] bytes = new byte[hex.length() / 2];
for (int i = 0; i < bytes.length; i++) {
bytes[i] = (byte) Integer.parseInt(hex.substring(i * 2, i * 2 + 2), 16);
}
return bytes;
}
How am I to interpret the line: int value = (high << 4) | low;?
look at this example for your last two digits (42):
int high = 4; // binary 0100
int low = 2; // binary 0010
int value = (high << 4) | low;
int value = (0100 << 4) | 0010; // shift 4 to left
int value = 01000000 | 0010; // bitwise or
int value = 01000010;
int value = 66; // 01000010 == 0x42 == 66
You can replace the << and | in this case with * and +, but I don't recommend it.
The expression
int value = (high << 4) | low;
is equivalent to
int value = high * 16 + low;
The subtraction of 256 to get a value between -128 and 127 is unnecessary. Simply casting, for example, 128 to a byte will produce the correct result. The lowest 8 bits of the int 128 have the same pattern as the byte -128: 0x80.
I'd write it simply as:
rawData[i] = (byte) ((high << 4) | low);
Is there any simpler way (preferably
without bit-handling) to do this
conversion?
You can use the Hex class in Apache commons, but internally, it will do the same thing, perhaps with minor differences.
How am I to interpret the line: int value = (high << 4) | low;?
This combines two hex digits, each of which represents 4 bits, into one unsigned 8-bit value stored as an int. The next two lines convert this to a signed Java byte.