I'm reading and writing to a ByteBuffer
import org.assertj.core.api.Assertions;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CharsetEncoder;
public class Solution{
public static void main(String[] args) throws Exception{
final CharsetEncoder messageEncoder = Charset.forName("ISO-8859-1").newEncoder();
String message = "TRANSACTION IGNORED";
String carrierName= "CARR00AB";
int messageLength = message.length()+carrierName.length()+8;
System.out.println(" --------Fill data---------");
ByteBuffer messageBuffer = ByteBuffer.allocate(4096);
messageBuffer.order(ByteOrder.BIG_ENDIAN);
messageBuffer.putInt(messageLength);
messageBuffer.put(messageEncoder.encode(CharBuffer.wrap(carrierName)));
messageBuffer.put(messageEncoder.encode(CharBuffer.wrap(message)));
messageBuffer.put((byte) 0x2b);
messageBuffer.flip();
System.out.println("------------Extract Data Approach 1--------");
CharsetDecoder messageDecoder = Charset.forName("ISO-8859-1").newDecoder();
int lengthField = messageBuffer.getInt();
System.out.println("lengthField="+lengthField);
int responseLength = lengthField - 12;
System.out.println("responseLength="+responseLength);
String messageDecoded= messageDecoder.decode(messageBuffer).toString();
System.out.println("messageDecoded="+messageDecoded);
String decodedCarrier = messageDecoded.substring(0, carrierName.length());
System.out.println("decodedCarrier="+ decodedCarrier);
String decodedBody = messageDecoded.substring(carrierName.length(), messageDecoded.length() - 1);
System.out.println("decodedBody="+decodedBody);
Assertions.assertThat(messageLength).isEqualTo(lengthField);
Assertions.assertThat(decodedBody).isEqualTo(message);
Assertions.assertThat(decodedBody).isEqualTo(message);
ByteBuffer messageBuffer2 = ByteBuffer.allocate(4096);
messageBuffer2.order(ByteOrder.BIG_ENDIAN);
messageBuffer2.putInt(messageLength);
messageBuffer2.put(messageEncoder.encode(CharBuffer.wrap(carrierName)));
messageBuffer2.put(messageEncoder.encode(CharBuffer.wrap(message)));
messageBuffer2.put((byte) 0x2b);
messageBuffer2.flip();
System.out.println("---------Extract Data Approach 2--------");
byte [] data = new byte[messageBuffer2.limit()];
messageBuffer2.get(data);
String dataString =new String(data, "ISO-8859-1");
System.out.println(dataString);
}
}
It works fine but then I thought to refactor it, Please see approach 2 in above code
byte [] data = new byte[messageBuffer.limit()];
messageBuffer.get(data);
String dataString =new String(data, "ISO-8859-1");
System.out.println(dataString);
Output= #CARR00ABTRANSACTION IGNORED+
Could you guys help me with explanation
why the integer is got missing in second approach while decoding ???
Is there any way to extract the integer in second approach??
Okay so you are trying to read an int from the Buffer which takes up 4 bits and then trying to get the whole data after reading 4 bits
What I have done is call messageBuffer2.clear(); after reading the int to resolve this issue. here is the full code
System.out.println(messageBuffer2.getInt());
byte[] data = new byte[messageBuffer2.limit()];
messageBuffer2.clear();
messageBuffer2.get(data);
String dataString = new String(data, StandardCharsets.ISO_8859_1);
System.out.println(dataString);
Output is:
35
#CARR0033TRANSACTION IGNORED+
Edit: So basically when you are calling clear it resets various variables and it also resets the position it's getting from and thats how it fixes it.
Related
I am using Java (JNA) to access DLL code. Here is the DLL I am using:
U32 ReadMemU32(U32 Addr, U32 NumItems, U32* pData, U8* pStatus);
Here is my sample code:
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.ptr.*;
import com.sun.jna.*;
import java.lang.*;
import java.util.*;
import com.sun.jna.*;
import java.lang.*;
public class DLL1 {
#UserFunction(Description="DLL1 Java wrapper", Name = "DLL1")
public static String DLL1(
#UserFunction.Arg(Name = "Address", Description = "Register Address")
String Address
) throws Exception {
byte Status[] = new byte[1];
PointerByReference Data = new PointerByReference ();
long Addr = Long.parseLong(Address.substring(2, Address.length()));
DllInterface.INSTANCE.ReadMemU32(Addr, 1, Data, Status);
System.out.println("Data = " + Data);
System.out.println("Data.getValue() = " + Data.getValue());
return null;
}
// C++ function definition and tell which DLL to fetch
public interface DllInterface extends Library {
DllInterface INSTANCE = (DllInterface) Native.loadLibrary("<dll name>", DllInterface.class);
long ReadMemU32(long Addr, long NumItems, PointerByReference pData, byte[] pStatus);
}
}
The result I am getting is:
Data = allocated#0xe25ac90 (8 bytes)(com.sun.jna.ptr.PointerByReference#e25ac90)
Data.getValue() = native#0xaaaaaaaa
The result (0xaaaaaaaa) does not match with what I expect it to be.
I am new to Java and I am not sure if PointerByReference is the correct thing to use for U32* pData. Please advice.
Looking at the type definition for the function, I would make an educated guess and say that pData is the array pointer to which the function writes the read memory. You can use int[] directly here. Since the status is returned as an out parameter via pointer, you can use a ByteByReference.
In Java, long refers to a 64-bit integer, equivalent to C++'s long long. Since the type is U32, it's probably safe to guess that it refers to a 32-bit integer. This would be an int. You can check JNA's type mappings here.
Notice also that since the function accepts a 32-bit integer for the address, you cannot pass 64-bit addresses to the function.
I would guess that one proper mapping for this function would be:
int ReadMemU32(
int address,
int numItems,
int[] pData,
ByteByReference status
);
If your function truly only needs to read one 32-bit integer from memory, you can change pData's type to IntByReference and use it in the same way as the status parameter's ByteByReference.
Since the function returns an integer, I would guess that it returns the amount of bytes read. If so, it's important to check that this value matches with what you're expecting. Check what the library's documentation says about the return value and error states.
Try this:
import java.lang.*;
import java.util.*;
import com.sun.jna.*;
import com.sun.jna.ptr.*;
public class DLL1 {
#UserFunction(Description="DLL1 Java wrapper", Name = "DLL1")
public static String DLL1(
#UserFunction.Arg(Name = "Address", Description = "Register Address")
String Address
) throws Exception {
String addressWithoutPrefix = Address.substring(2)
int parsedAddress = Integer.parseInt(addressWithoutPrefix, 16)
int bytesToRead = 1;
int[] buffer = new int[bytesToRead];
ByteByReference status = new ByteByReference(0);
int BytesRead = DllInterface.INSTANCE.ReadMemU32(
parsedAddress,
bytesToRead,
buffer,
status
);
System.out.println("Status = " + status.getValue());
System.out.println("Bytes read = " + bytesRead);
System.out.println("Data = " + Arrays.toString(buffer));
return null;
}
// C++ function definition and tell which DLL to fetch
public interface DllInterface extends Library {
DllInterface INSTANCE = (DllInterface) Native.loadLibrary("<dll name>", DllInterface.class);
int ReadMemU32(
int address,
int numItems,
int[] pData,
ByteByReference status
);
}
}
I need to get big Boolean arrays or BitSets from Java into Python via a text file. Ideally I want to go via a Base64 representation to stay compact, but still be able to embed the value in a CSV file. (So the boolean array will be one column in a CSV file.)
However I am having issues to get the byte alignment right. Where/how should I specify the correct byte order?
This is one example, working in the sense that it executes but not working in that my bits aren't where I want them.
Java:
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.util.Base64;
import java.util.Base64.Encoder;
import java.util.BitSet;
public class basictest {
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
Encoder b64 = Base64.getEncoder();
String name = "name";
BitSet b = new BitSet();
b.set(444);
b.set(777);
b.set(555);
byte[] bBytes = b.toByteArray();
String fp_str = b64.encodeToString(bBytes);
BufferedWriter w = new BufferedWriter(new FileWriter("out.tsv"));
w.write(name + "\t" + fp_str + "\n");
w.close();
}
}
Python:
import numpy as np
import base64
from bitstring import BitArray, BitStream ,ConstBitStream
filename = "out.tsv"
with open(filename) as file:
data = file.readline().split('\t')
b_b64 = data[1]
b_bytes = base64.b64decode(b_b64)
b_bits = BitArray(bytes=b_bytes)
b_bits[444] # False
b_bits[555] # False
b_bits[777] # False
# but
b_bits[556] # True
# it's not shifted:
b_bits[445] # False
I am now reversing the bits in every byte using https://stackoverflow.com/a/5333563/1259675:
numbits = 8
r_bytes = [
sum(1<<(numbits-1-i) for i in range(numbits) if b>>i&1)
for b in b_bytes]
b_bits = BitArray(r_bytes)
This works, but is there a method that doesn't involve myself fiddling with the bits?
If:
the maximum bit to set is "sufficiently small".
and the data, you want to encode doesn't vary in size too much.
..then one approach can be:
Set max (+ min) significant bit(s in java) .
and ignore them in python .
, then it c(sh!)ould work without byte reversal, or further transformation:
// assuming a 1024 bit word
public static final int LEFT_SIGN = 0;
public static final int RIGHT_SIGN = 1025; //choose a size, that fits your needs [0 .. Integer.MAX_VALUE - 1 (<-theoretically)]
public static void main(String[] args) throws Exception {
...
b.set(LEFT_SIGN);
b.set(444 + 1);
b.set(777 + 1);
b.set(555 + 1);
b.set(RIGHT_SIGN);
...
and then in python:
# as before ..
b_bits[0] # Ignore!
b_bits[445] # True
b_bits[556] # True
b_bits[778] # True
b_bits[1025] # Ignore!;)
Your convenience (= encoding) 'd be the (maximum) "word length" ... with all its benefits and drawbacks.
We can use the bitarray package from python for this particular usecase.
from bitarray import bitarray
import base64
with open(filename) as file:
data = file.readline().strip().split('\t')
b_b64 = data[1]
b_bytes = base64.b64decode(b_b64)
bs = bitarray(endian='little')
bs.frombytes(b_bytes)
print bs
I implemented a Java pipeline which stores some objects in postgres (9.5).
Objects are first serialized to a json string with jackson (2.5.4) and then sent with JDBC to the DBMS in a JSON column type.
When needed objects are read from database, deserialized and used by the application.
The problem is that when i try to query my data with psql I have some strange error, linked with UTF-8:
kronos=# select payload from messages where payload->>'errorMessage' = 'some error';
ERROR: unsupported Unicode escape sequence
DETAIL: \u0000 cannot be converted to text.
CONTEXT: JSON data, line 1:
...globalIdResolution":"GENERATED","authorFullName":...
I am not particularly worried, since my application can write and read all the data in the RDBMS. Is this a psql issue or maybe I am doing something wrong? Is there a way to be sure that my data should be accessible from any application, also if not java+jackson based?
I think this couldn't be jackson's fault, since I tried with a unit test but the string doens't seem to contain \0000 char:
package it.celi.dd.kronos.json;
import static org.junit.Assert.*;
import java.util.HashMap;
import java.util.Map;
import org.junit.Test;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JacksonUTF8Test {
#Test
public void test() throws JsonProcessingException {
final ObjectMapper objectMapper = new ObjectMapper();
final String utf8 = "؏ـــطــہ";
final Map<String, Object> map = new HashMap<>();
map.put("content", txt(1600) + utf8 + txt(500));
final String json = objectMapper.writeValueAsString(map);
final int jsonBoilerplate = "{\"content\":\"".length() + "\"}".length();
assertEquals(1600 + utf8.length() + 500 + jsonBoilerplate, json.length());
System.out.println(json.length());
assertEquals("{\"content\":\"" + txt(1600) + utf8 + txt(500) + "\"}", json);
}
private String txt(final int chars) {
final StringBuilder builder = new StringBuilder();
for(int i = 0; i < chars; i ++) {
builder.append('a');
}
return builder.toString();
}
}
I have the following code which merges two audio files into one:
import java.io.File;
import java.io.IOException;
import java.io.SequenceInputStream;
import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
public class WavAppender {
public static void main(String[] args) {
String wavFile1 = "D:\\wav1.wav";
String wavFile2 = "D:\\wav2.wav";
try {
AudioInputStream clip1 = AudioSystem.getAudioInputStream(new File(wavFile1));
AudioInputStream clip2 = AudioSystem.getAudioInputStream(new File(wavFile2));
AudioInputStream appendedFiles =
new AudioInputStream(
new SequenceInputStream(clip1, clip2),
clip1.getFormat(),
clip1.getFrameLength() + clip2.getFrameLength());
AudioSystem.write(appendedFiles,
AudioFileFormat.Type.WAVE,
new File("D:\\wavAppended.wav"));
} catch (Exception e) {
e.printStackTrace();
}
}
}
I will have a string in the following format [1,2,3,4,5]. Based on the string I will need to select the appropriate wav file. For example if the string is in the format [3,4,5,6,7], I will need to send wavfile 3, wavfile4, wavfile5, wavfile6 and wavfile 7. What is the best way to achieve this?
Create an array or List of items, so that wavfile1 is at index 0, wavfile2 is at index 1 and so on and so forth.
Take each element from the String array and convert to to int, subtract one from it (as arrays and lists are zero indexed) and that becomes your index for the "wave file array"...
String waveFile = waveFiles[Integer.parseInt(indicies[0]) - 1];
...Now this prone to some issues, particularly the conversion of the String to int...
Instead, you could use a Map instead, where each wave file is mapped to the corresponding String id
Map<String, String> waveFiles = new ...;
waveFiles.put("1", "WaveFile1");
waveFiles.put("2", "WaveFile2");
//...
Then you would simply use the value from the String array to look it up...
String waveFile = waveFiles.get(indicies[0]);
As some ideas...
Take a look at the Collections Trail for more details and ideas...
I am using the Java fftw3 wrapper taken from this question. (Code here)
I just wanted to apply a 2nd type DCT transform to an array of double elements, but I keep getting this error if i try to call the fftw_execute method:
java(787,0x10b243000) malloc: *** error for object 0x7fba642c5408: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Why?
Here's my code:
package com.project.fftw3;
import java.io.File;
import java.io.IOException;
import java.nio.DoubleBuffer;
import fftw3.FFTW3Library;
import fftw3.FFTW3Library.fftw_plan;
public class MainClass {
static FFTW3Library fftw = FFTW3Library.INSTANCE;
public static void main(String[] args) {
int i,j,w,h;
File in = new File("Images/Baboon.bmp");
//File out = new File("Baboon-" + System.currentTimeMillis() + ".txt");
try {
ImageMatrix im = new ImageMatrix(in);
w=im.getImageWidth();
h=im.getImageHeight();
double [] row = im.getRow(0);
double [] oarr = new double[w];
DoubleBuffer din = DoubleBuffer.wrap(row);
DoubleBuffer dout = DoubleBuffer.wrap(oarr);
fftw_plan p = fftw.fftw_plan_dft_1d(din.array().length,din,dout,5,FFTW3Library.FFTW_ESTIMATE); //5 is REDFT10
fftw.fftw_execute(p);
fftw.fftw_destroy_plan(p);
} catch (IOException e) {
}
}
}
It looks like you have the wrong dimension here:
double [] oarr = new double[h];
Change this to:
double [] oarr = new double[w];
This is consistent with the error you are seeing, assuming w > h.