My Java program needs to send a binary payload via QR Code, but I can't get it to work. I have tried several QR Code libraries and many approaches, but all seem to have this problem. My current implementation uses ZXING.
The problem is that all the Java libraries I've tried seem to be focused on String payloads, and do not provide support for binary data. The common suggested solution to this is to encode the binary data as Base64. However, my data is already near the size limit of QR Codes. With the 33% inflation caused by Base64 encoding, my data is too big. I have already expended significant effort into reducing the size of the payload, and it currently consists of 4 character hashes delimited by new lines; all inside max level compression by the Java Deflator class. I can't make it any smaller.
I need a way to store binary data in a QR code with minimal data inflation overhead.
Update:
I recently went back and published the referenced code as a project on GitHub for anyone who wants to use it.
https://github.com/yurelle/Base45Encoder
I developed a solution which only introduces a storage efficiency loss of -8%. It exploits a built-in compression optimization of the ZXING QR Code Library.
Explanation
ZXING will automatically detect if your String payload is purely AlphaNumeric (by their own definition), and if so, it will automatically compress 2 AlphaNumeric characters into 11 bits. The definition ZXING uses for "alphanumeric" is all-caps only, 0-9, and a few special symbols ('/', ':', etc.). All told, their definition allows 45 possible values. Then, it packs 2 of these Base45 digits into 11 bits.
2 digits in base 45 is 2,025 possible values. 11 bits has a maximum storage capacity of 2,048 possible states. This is only a loss of 1.1% in storage efficiency behind raw binary.
45 ^ 2 = 2,025
2 ^ 11 = 2,048
2,048 - 2,025 = 23
23 / 2,048 = 0.01123046875 = 1.123%
However, this is the ideal / theoretical efficiency. My implementation processes data in chunks, using a Long as a computational buffer. However, since Java Long's are singed, we can only use the lower 7 bytes. The conversion code requires continuously positive values; using the highest 8th byte would contaminate the sign bit and randomly produce negative values.
Real-World Test:
Using a 7 byte Long to encode a 2KB buffer of random bytes, we get the following results.
Raw Binary Size: 2,048
Encoded String Size: 3,218
QR Code Alphanum Size: 2,213 (after the QR Code compresses 2 base45 digits to 11 bits)
This is a real-world storage efficiency loss of only 8%.
2,213 - 2,048 = 165
165 / 2,048 = 0.08056640625 = 8.0566%
Solution
I implemented it as a self-contained static utility class, so all you have to do is call:
//Encode
final byte[] myBinaryData = ...;
final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(myBinaryData);
//Decode
final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(encodedStr);
Alternatively, you can also do it via InputStreams:
//Encode
final InputStream in_1 = ... ;
final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(in_1);
//Decode
final InputStream in_2 = ... ;
final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(in_2);
Here's the implementation
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.Map;
/**
* For some reason none of the Java QR Code libraries support binary payloads. At least, none that
* I could find anyway. The commonly suggested workaround for this is to use Base64 encoding.
* However, this results in a 33% payload size inflation. If your payload is already near the size
* limit of QR codes, this is a lot.
*
* This class implements an encoder which takes advantage of a built-in compression optimization
* of the ZXING QR Code library, to enable the storage of Binary data into a QR Code, with a
* storage efficiency loss of only -8%.
*
* The built-in optimization is this: ZXING will automatically detect if your String payload is
* purely AlphaNumeric (by their own definition), and if so, it will automatically compress 2
* AlphaNumeric characters into 11 bits.
*
*
* ----------------------
*
*
* The included ALPHANUMERIC_TABLE is the conversion table used by the ZXING library as a reverse
* index for determining if a given input data should be classified as alphanumeric.
*
* See:
*
* com.google.zxing.qrcode.encoder.Encoder.chooseMode(String content, String encoding)
*
* which scans through the input string one character at a time and passes them to:
*
* getAlphanumericCode(int code)
*
* in the same class, which uses that character as a numeric index into the the
* ALPHANUMERIC_TABLE.
*
* If you examine the values, you'll notice that it ignores / disqualifies certain values, and
* effectively converts the input into base 45 (0 -> 44; -1 is interpreted by the calling code
* to mean a failure). This is confirmed in the function:
*
* appendAlphanumericBytes(CharSequence content, BitArray bits)
*
* where they pack 2 of these base 45 digits into 11 bits. This presents us with an opportunity.
* If we can take our data, and convert it into a compatible base 45 alphanumeric representation,
* then the QR Encoder will automatically pack that data into sub-byte chunks.
*
* 2 digits in base 45 is 2,025 possible values. 11 bits has a maximum storage capacity of 2,048
* possible states. This is only a loss of 1.1% in storage efficiency behind raw binary.
*
* 45 ^ 2 = 2,025
* 2 ^ 11 = 2,048
* 2,048 - 2,025 = 23
* 23 / 2,048 = 0.01123046875 = 1.123%
*
* However, this is the ideal / theoretical efficiency. This implementation processes data in
* chunks, using a Long as a computational buffer. However, since Java Long's are singed, we
* can only use the lower 7 bytes. The conversion code requires continuously positive values;
* using the highest 8th byte would contaminate the sign bit and randomly produce negative
* values.
*
*
* Real-World Test:
*
* Using a 7 byte Long to encode a 2KB buffer of random bytes, we get the following results.
*
* Raw Binary Size: 2,048
* Encoded String Size: 3,218
* QR Code Alphanum Size: 2,213 (after the QR Code compresses 2 base45 digits to 11 bits)
*
* This is a real-world storage efficiency loss of only 8%.
*
* 2,213 - 2,048 = 165
* 165 / 2,048 = 0.08056640625 = 8.0566%
*/
public class BinaryToBase45Encoder {
public final static int[] ALPHANUMERIC_TABLE;
/*
* You could probably just copy & paste the array literal from the ZXING source code; it's only
* an array definition. But I was unsure of the licensing issues with posting it on the internet,
* so I did it this way.
*/
static {
final Field SOURCE_ALPHANUMERIC_TABLE;
int[] tmp;
//Copy lookup table from ZXING Encoder class
try {
SOURCE_ALPHANUMERIC_TABLE = com.google.zxing.qrcode.encoder.Encoder.class.getDeclaredField("ALPHANUMERIC_TABLE");
SOURCE_ALPHANUMERIC_TABLE.setAccessible(true);
tmp = (int[]) SOURCE_ALPHANUMERIC_TABLE.get(null);
} catch (NoSuchFieldException e) {
e.printStackTrace();//Shouldn't happen
tmp = null;
} catch (IllegalAccessException e) {
e.printStackTrace();//Shouldn't happen
tmp = null;
}
//Store
ALPHANUMERIC_TABLE = tmp;
}
public static final int NUM_DISTINCT_ALPHANUM_VALUES = 45;
public static final char[] alphaNumReverseIndex = new char[NUM_DISTINCT_ALPHANUM_VALUES];
static {
//Build AlphaNum Index
final int len = ALPHANUMERIC_TABLE.length;
for (int x = 0; x < len; x++) {
// The base45 result which the alphanum lookup table produces.
// i.e. the base45 digit value which String characters are
// converted into.
//
// We use this value to build a reverse lookup table to find
// the String character we have to send to the encoder, to
// make it produce the given base45 digit value.
final int base45DigitValue = ALPHANUMERIC_TABLE[x];
//Ignore the -1 records
if (base45DigitValue > -1) {
//The index into the lookup table which produces the given base45 digit value.
//
//i.e. to produce a base45 digit with the numeric value in base45DigitValue, we need
//to send the Encoder a String character with the numeric value in x.
alphaNumReverseIndex[base45DigitValue] = (char) x;
}
}
}
/*
* The storage capacity of one digit in the number system; i.e. the maximum
* possible number of distinct values which can be stored in 1 logical digit
*/
public static final int QR_PAYLOAD_NUMERIC_BASE = NUM_DISTINCT_ALPHANUM_VALUES;
/*
* We can't use all 8 bytes, because the Long is signed, and the conversion math
* requires consistently positive values. If we populated all 8 bytes, then the
* last byte has the potential to contaminate the sign bit, and break the
* conversion math. So, we only use the lower 7 bytes, and avoid this problem.
*/
public static final int LONG_USABLE_BYTES = Long.BYTES - 1;
//The following mapping was determined by brute-forcing -1 Long (all bits 1), and compressing to base45 until it hit zero.
public static final int[] BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION = new int[] {0,2,3,5,6,8,9,11,12};
public static final int NUM_BASE45_DIGITS_PER_LONG = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[LONG_USABLE_BYTES];
public static final Map<Integer, Integer> BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION = new HashMap<>();
static {
//Build Reverse Lookup
int len = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION.length;
for (int x=0; x<len; x++) {
int numB45Digits = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[x];
BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION.put(numB45Digits, x);
}
}
public static String encodeToBase45QrPayload(final byte[] inputData) throws IOException {
return encodeToBase45QrPayload(new ByteArrayInputStream(inputData));
}
public static String encodeToBase45QrPayload(final InputStream in) throws IOException {
//Init conversion state vars
final StringBuilder strOut = new StringBuilder();
int data;
long buf = 0;
// Process all input data in chunks of size LONG.BYTES, this allows for economies of scale
// so we can process more digits of arbitrary size before we hit the wall of the binary
// chunk size in a power of 2, and have to transmit a sub-optimal chunk of the "crumbs"
// left over; i.e. the slack space between where the multiples of QR_PAYLOAD_NUMERIC_BASE
// and the powers of 2 don't quite line up.
while(in.available() > 0) {
//Fill buffer
int numBytesStored = 0;
while (numBytesStored < LONG_USABLE_BYTES && in.available() > 0) {
//Read next byte
data = in.read();
//Push byte into buffer
buf = (buf << 8) | data; //8 bits per byte
//Increment
numBytesStored++;
}
//Write out in lower base
final StringBuilder outputChunkBuffer = new StringBuilder();
final int numBase45Digits = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[numBytesStored];
int numB45DigitsProcessed = 0;
while(numB45DigitsProcessed < numBase45Digits) {
//Chunk out a digit
final byte digit = (byte) (buf % QR_PAYLOAD_NUMERIC_BASE);
//Drop digit data from buffer
buf = buf / QR_PAYLOAD_NUMERIC_BASE;
//Write Digit
outputChunkBuffer.append(alphaNumReverseIndex[(int) digit]);
//Track output digits
numB45DigitsProcessed++;
}
/*
* The way this code works, the processing output results in a First-In-Last-Out digit
* reversal. So, we need to buffer the chunk output, and feed it to the OutputStream
* backwards to correct this.
*
* We could probably get away with writing the bytes out in inverted order, and then
* flipping them back on the decode side, but just to be safe, I'm always keeping
* them in the proper order.
*/
strOut.append(outputChunkBuffer.reverse().toString());
}
//Return
return strOut.toString();
}
public static byte[] decodeBase45QrPayload(final String inputStr) throws IOException {
//Prep for InputStream
final byte[] buf = inputStr.getBytes();//Use the default encoding (the same encoding that the 'char' primitive uses)
return decodeBase45QrPayload(new ByteArrayInputStream(buf));
}
public static byte[] decodeBase45QrPayload(final InputStream in) throws IOException {
//Init conversion state vars
final ByteArrayOutputStream out = new ByteArrayOutputStream();
int data;
long buf = 0;
int x=0;
// Process all input data in chunks of size LONG.BYTES, this allows for economies of scale
// so we can process more digits of arbitrary size before we hit the wall of the binary
// chunk size in a power of 2, and have to transmit a sub-optimal chunk of the "crumbs"
// left over; i.e. the slack space between where the multiples of QR_PAYLOAD_NUMERIC_BASE
// and the powers of 2 don't quite line up.
while(in.available() > 0) {
//Convert & Fill Buffer
int numB45Digits = 0;
while (numB45Digits < NUM_BASE45_DIGITS_PER_LONG && in.available() > 0) {
//Read in next char
char c = (char) in.read();
//Translate back through lookup table
int digit = ALPHANUMERIC_TABLE[(int) c];
//Shift buffer up one digit to make room
buf *= QR_PAYLOAD_NUMERIC_BASE;
//Append next digit
buf += digit;
//Increment
numB45Digits++;
}
//Write out in higher base
final LinkedList<Byte> outputChunkBuffer = new LinkedList<>();
final int numBytes = BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION.get(numB45Digits);
int numBytesProcessed = 0;
while(numBytesProcessed < numBytes) {
//Chunk out 1 byte
final byte chunk = (byte) buf;
//Shift buffer to next byte
buf = buf >> 8; //8 bits per byte
//Write byte to output
//
//Again, we need to invert the order of the bytes, so as we chunk them off, push
//them onto a FILO stack; inverting their order.
outputChunkBuffer.push(chunk);
//Increment
numBytesProcessed++;
}
//Write chunk buffer to output stream (in reverse order)
while (outputChunkBuffer.size() > 0) {
out.write(outputChunkBuffer.pop());
}
}
//Return
out.flush();
out.close();
return out.toByteArray();
}
}
Here are some tests I ran to verify the code:
#Test
public void stringEncodingTest() throws IOException {
//Init test data
final String testStr = "Some cool input data! !##$%^&*()_+";
//Encode
final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(testStr.getBytes("UTF-8"));
//Decode
final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(encodedStr);
final String decodedStr = new String(decodedBytes, "UTF-8");
//Output
final boolean matches = testStr.equals(decodedStr);
assert(matches);
System.out.println("They match!");
}
#Test
public void binaryEncodingAccuracyTest() throws IOException {
//Init test data
final int maxBytes = 10_000;
for (int x=1; x<=maxBytes; x++) {
System.out.print("x: " + x + "\t");
//Encode
final byte[] inputArray = getTestBytes(x);
final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(inputArray);
//Decode
final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(encodedStr);
//Output
for (int y=0; y<x; y++) {
assertEquals(inputArray[y], decodedBytes[y]);
}
System.out.println("Passed!");
}
}
#Test
public void binaryEncodingEfficiencyTest() throws IOException, WriterException, NoSuchMethodException, InvocationTargetException, IllegalAccessException {
//Init test data
final byte[] inputData = new byte[2048];
new Random().nextBytes(inputData);
//Encode
final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(inputData);
//Write to QR Code Encoder // Have to use Reflection to force access, since the function is not public.
final BitArray qrCode = new BitArray();
final Method appendAlphanumericBytes = com.google.zxing.qrcode.encoder.Encoder.class.getDeclaredMethod("appendAlphanumericBytes", CharSequence.class, BitArray.class);
appendAlphanumericBytes.setAccessible(true);
appendAlphanumericBytes.invoke(null, encodedStr, qrCode);
//Output
final int origSize = inputData.length;
final int qrSize = qrCode.getSizeInBytes();
System.out.println("Raw Binary Size:\t\t" + origSize + "\nEncoded String Size:\t" + encodedStr.length() + "\nQR Code Alphanum Size:\t" + qrSize);
//Calculate Storage Efficiency Loss
final int delta = origSize - qrSize;
final double efficiency = ((double) delta) / origSize;
System.out.println("Storage Efficiency Loss: " + String.format("%.3f", efficiency * 100) + "%");
}
public static byte[] getTestBytes(int numBytes) {
final Random rand = new Random();
final ByteArrayOutputStream bos = new ByteArrayOutputStream();
for (int x=0; x<numBytes; x++) {
//bos.write(255);// -1 (byte) = 255 (int) = 1111 1111
byte b = (byte) rand.nextInt();
bos.write(b);
}
return bos.toByteArray();
}
Related
below is a code from arduino:
typedef union{
float number;
uint8_t bytes[4];
} FLOATUNION_t;
FLOATUNION_t myValue;
float getFloat(){
int cont = 0;
FLOATUNION_t f;
while (cont < 4 ){
f.bytes[cont] = Serial.read() ;
cont = cont +1;
}
return f.number;
}
is there any alternative to this for processing programming(java)?
totally i need a code which recieve binary data from serial port and i want that data in float type in processing programming.
originally i am sending some data through serial send block in Simulink to processing programming, i am using processing.serial library , but i could not get thise bits and convert them to float or int.
You need to :
buffer each byte a time from Simulink into Processing: Serial's buffer() / serialEvent() / readBytes(bytesFromSimulink) could work nicely in tandem here
pack the bytes into a an int (shifting bytes as needed) and OR-ing them:
int intBits = bytesFromSimulink[3] << 24 | bytesFromSimulink[2] << 16 | bytesFromSimulink[1] << 8 | bytesFromSimulink[0];
convert the int to a float via Float.intBitsToFloat(): floatFromSimulink = Float.intBitsToFloat( intBits );
Here's a basic sketch to illustrate the ideas above:
import processing.serial.*;
// how many bytes are expecting sent in one go
final int SERIAL_BUFFER_SIZE = 4;
// pre-allocate serial read buffer
byte[] bytesFromSimulink = new byte[SERIAL_BUFFER_SIZE];
// float from bytes
float floatFromSimulink;
// serial port reference
final String PORT_NAME = "COM2";
final int BAUD_RATE = 115200;
Serial simulinkPort;
void setup(){
size(300, 300);
try{
simulinkPort = new Serial(this, PORT_NAME, BAUD_RATE);
// only fire serialEvent() when the right number of bytes has been buffered
simulinkPort.buffer(SERIAL_BUFFER_SIZE);
}catch(Exception e){
println("error opening serial port(" + PORT_NAME + "): double check the port name, wiring and make sure the port isn't already open in another application");
e.printStackTrace();
exit();
}
}
void draw(){
background(0);
// format bytes to hex and float to 2 decimal places
text(String.format("hex: %s\nfloat: %.2f", hex(byteFromSimulink), floatFromSimulink),
10, 15);
}
void serialEvent(Serial port) {
port.readBytes(bytesFromSimulink);
// pack bytes into a 32bit int (shifting each byte accordingly): double check the byte order (e.g. LSB / MSB)
int intBits = bytesFromSimulink[3] << 24 |
bytesFromSimulink[2] << 16 |
bytesFromSimulink[1] << 8 |
bytesFromSimulink[0];
// convert int to to float
floatFromSimulink = Float.intBitsToFloat( intBits );
}
// pretty-print byte array
String hex(byte[] data){
String output = "";
for(byte singleByte : data){
output += hex(singleByte) + ' ';
}
return output;
}
Hopefully the above just works, but bare in mind it's untested code.
There are two things I could think could go wrong:
The bytes not arriving in the correct order. (let's say Simulink is continuously streaming serial data, but Processing starts later and only catches data from the 2nd, 3rd or 4th byte instead of the first: data will be shifted). You can try to remove buffer()/serialEvent() with a blocking loop and getting one byte at a time (e.g. if(simulinkPort.available() >= 1) myNewByte = simulinkPort.read();) and counting / packing bytes into the byte array manually. You could also try a call/response approach: e.g. Simulink doesn't send any data until it receives a single character from Processing (let's say 'A'), then starts streaming, so Processing is ready to buffer 4 bytes at a time from the get go.
I'm not sure in which order the bytes are sent from simulink: above I'm assuming right to left, but it's the other way around simply swap indices: int intBits = byteFromSimulink[0] << 24 | byteFromSimulink[1] << 16 | byteFromSimulink[2] << 8 | byteFromSimulink[3];
The other gotcha in Java/Processing is that bytes are from -127 to 127 so you would want mask bytes when inspecting: println(myByte & 0xFF);
Based on g00se's suggestion in the comments bellow here's an attempt at a ByteBuffer option:
import java.nio.ByteBuffer;
import processing.serial.*;
// how many bytes are expecting sent in one go
final int SERIAL_BUFFER_SIZE = 4;
// pre-allocate serial read buffer
ByteBuffer bytesFromSimulink;
// float from bytes
float floatFromSimulink;
// serial port reference
final String PORT_NAME = "COM2";
final int BAUD_RATE = 115200;
Serial simulinkPort;
void setup(){
size(300, 300);
try{
simulinkPort = new Serial(this, PORT_NAME, BAUD_RATE);
// only fire serialEvent() when the right number of bytes has been buffered
simulinkPort.buffer(SERIAL_BUFFER_SIZE);
bytesFromSimulink = ByteBuffer.allocate(SERIAL_BUFFER_SIZE);
}catch(Exception e){
println("error opening serial port(" + PORT_NAME + "): double check the port name, wiring and make sure the port isn't already open in another application");
e.printStackTrace();
exit();
}
}
void draw(){
background(0);
// format bytes to hex and float to 2 decimal places
text(String.format("hex: %s\nfloat: %.2f", hex(bytesFromSimulink), floatFromSimulink),
10, 15);
}
void serialEvent(Serial port) {
// pass new data to the byte buffer
bytesFromSimulink.put(port.readBytes(SERIAL_BUFFER_SIZE));
// set the index back to 0
bytesFromSimulink.rewind();
// read the first (rewinded) 4 bytes as a float
floatFromSimulink = bytesFromSimulink.getFloat();
}
// pretty-print byte array
String hex(ByteBuffer data){
String output = "";
for(int i = 0 ; i < data.limit(); i++){
output += hex(data.get(i)) + ' ';
}
return output;
}
I have a multichannel input (i'm using Soundflower 64ch on mac), and I'm trying to mixdown 4 channels of the 64 channels to an stereo output.
What i am doing is, reading chunks of 1024 frames, with 64 channels every frame, then converting the bytebuffer to Short array (values between -32,768 <-> 32,767, because samples are 16 bits).
This way I add for example channel1[sample] + channel2[sample] and I get the mix of both channels.
But here is a problem, the sum can overflow the Short (16 bit) range, introducing saturation in the sound. So what I'm doing is (channel1[sample] + channel2[sample]) / 2 but when I divide by 2, I hear a lot of white sound.
Also if I try to reduce the volumen of a channel by doing channel1[sample] * 0.5 there is a lot of saturation.
Why does it happen?
Here is my full code, note that I'm converting bytes to short to handle better, and then I'm converting back to bytes for write the mix to the stereo output:
public static void main(String[] args) throws LineUnavailableException {
int inputChannels = 64;
AudioFormat inputFormat = new AudioFormat(48000, 16, inputChannels, true, false);
AudioFormat outputFormat = new AudioFormat(48000, 16, 2, true, false);
TargetDataLine mic = AudioSystem.getTargetDataLine(inputFormat);
SourceDataLine speaker = AudioSystem.getSourceDataLine(outputFormat);
mic.open(inputFormat);
speaker.open(outputFormat);
mic.start();
speaker.start();
AudioInputStream audioInputStream = new AudioInputStream(mic);
int bytesPerFrame = audioInputStream.getFormat().getFrameSize();
// Set an arbitrary buffer size of 1024 frames.
int CHUNK = 1024 ;
int numBytes = CHUNK * bytesPerFrame;
byte[] audioBytes = new byte[numBytes];
try {
byte[][] frames = new byte[CHUNK][bytesPerFrame];
int i = 0, j = 0
;
while (true) {
// read to audioBytes.
audioInputStream.read(audioBytes);
// split audioBytes in _CHUNK_ frames (1024 frames)
for(j=0; j<CHUNK; j++) {
frames[j] = Arrays.copyOfRange(audioBytes, j * bytesPerFrame, j * bytesPerFrame + bytesPerFrame);
}
// convert bytearray to shortarray
short[][] shortFrames = new short[CHUNK][inputChannels];
for(i=0; i < frames.length; i++) {
ByteBuffer.wrap(frames[i]).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shortFrames[i]);
}
short[] leftOutput = new short[CHUNK*2];
short[] rightOutput = new short[CHUNK*2];
for (i=0; i<CHUNK; i++) {
short channel1 = shortFrames[i][0];
short channel2 = shortFrames[i][1];
short channel3 = shortFrames[i][2];
short channel4 = shortFrames[i][3];
leftOutput[i] = (short)(channel4);
rightOutput[i] = (short)(channel4);;
}
//convert shortarray in byte buffer
ByteBuffer byteBuf = ByteBuffer.allocate(CHUNK * 2 * 2); // 2 bytes * 2 output channels
for (i=0; i<CHUNK; i++) {
byteBuf.putShort(leftOutput[i]);
byteBuf.putShort(rightOutput[i]);
}
speaker.write(byteBuf.array(),0,byteBuf.array().length);
}
} catch (Exception ex) {
// Handle the error...
System.out.println("exception");
System.out.println(ex.toString());
}
}
IDK if the issue is how the bytes are being converted to shorts and back, but since you asked about this in the comment, I will post it. Assume buffer has contiguous little-endian bytes at 16-bit encoding. Just reverse the byte indexes for big-endian.
pcmShort = ( buffer[i] & 0xff ) | ( buffer[i+1] << 8 );
The conversion of pcm to byte that I use follows (for little-endian, reverse the indexes for big-endian):
outBuffer[i] = (byte)pcmShort[0];
outBuffer[i+1] = (byte)((int)pcmShort[0] >> 8);
Maybe you can use the two methods (your attempt with ByteBuffer and getShort, and the above) side-by-side on the same data and check if the resulting arrays hold the same values?
Another thing I'd try to do is to just get a single track working. If that sounds okay, then check on the mixing. It's kind of unlikely that the signals are so hot that they are overrunning. So something else is probably going on.
I should try this out myself, I'm not sure when I'll get to it. It could potentially be an improvement over what I've been doing.
I did test mix two PCM audio file.
but don't get true audio file.
I used this example
So, my code:
private void mixSound() throws IOException {
byte[] music1 = null;
music1 = new byte[in1.available()];
music1 = convertStreamToByteArray(in1);
in1.close();
byte[] music2 = null;
music2 = new byte[in2.available()];
music2 = convertStreamToByteArray(in2);
in2.close();
byte[] output = new byte[music1.length];
for (int i = 0; i < output.length; i++) {
samplef1 = music1[i] / 128.0f;
samplef2 = music2[i] / 128.0f;
float mixed = samplef1 + samplef2;
// reduce the volume a bit:
mixed *= 0.8;
// hard clipping
if (mixed > 1.0f) mixed = 1.0f;
if (mixed < -1.0f) mixed = -1.0f;
byte outputSample = (byte) (mixed * 128.0f);
output[i] = outputSample;
} //for loop
save = openFileOutput(filename, Context.MODE_PRIVATE);
save.write(output);
save.flush();
save.close();
}
public byte[] convertStreamToByteArray(InputStream is) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buff = new byte[8000];
int i;
while ((i = is.read(buff, 0, buff.length)) > 0) {
baos.write(buff, 0, i);
}
return baos.toByteArray(); // be sure to close InputStream in calling function
}
2 audio files with bit rate 64000 & sampling rate 16000 GH & sterio
in1 = getResources().openRawResource(R.raw.a_2);
in2 = getResources().openRawResource(R.raw.a_diz_2);
Also try to convert
bytes array to short array -> then calculate-> then convert short to byte using converts methods
like bytes2Shorts(byte[] buf) and shorts2Bytes(short[] s).
But steel have a fail result.
Someone can say me Where is my wrong?
There are a number of issues here and I'll try to address some of them
First, using byte[] suggests that your PCM wave data format is AudioFormat.ENCODING_PCM_8BIT (or it should be this format if it already isn't). This format uses 8-bit (1 byte) unsigned, which means that the sound
samples are stored in the [0, 255] range (not in the [-127, +128] or [-128,+127] range).
This means that the negative values are in the [0, 127] range and the positive samples are in the [128,255] range.
When mixing values, it's best to prevent clipping right from the start so I'd use
byte mixed = (music1[i] + music2[i])/2; //this ensures that mixed remains within the `correct range` for your PCM format
You can also divide your samples by 128 (if you want to convert them to floating point values)
float samplef1 = (((float)music1[i]-127)/128 ; //converting samples to [-1, +1] range -- -1 corresponds a sample value of 0 and +1 to 255
float samplef2 = (((float)music2[i]-127)/128;
float mixed = (samplef1+samplef2)/2;
Note that you now have 2 options to play data(samples) generated in this way. Either, convert floats back to bytes or use the AudioFormat.ENCODING_PCM_FLOAT format.
audio files with bit rate 64000 & sampling rate 16000 GH & sterio
This can't be correct. Typical sampling rates are 4000Hz, 8000Hz, 11000Hz, 16000Hz, 22050Hz or 44100Hz. For bit depths, audio usually uses 8 bits, 16 bits or 32 bits.
For instance, CD quality audio uses 44100Hz, 16bit, stereo format.
I have a program in C# .net which writes 1 integer and 3 strings to a file, using BinaryWriter.Write().
Now I am programming in Java (for Android, and I'm new in Java), and I have to access the data which were previously written to a file using C#.
I tried using DataInputStream.readInt() and DataInputStream.readUTF(), but I can't get proper results. I usually get a UTFDataFormatException:
java.io.UTFDataFormatException: malformed input around byte 21
or the String and int I get is wrong...
FileInputStream fs = new FileInputStream(strFilePath);
DataInputStream ds = new DataInputStream(fs);
int i;
String str1,str2,str3;
i=ds.readInt();
str1=ds.readUTF();
str2=ds.readUTF();
str3=ds.readUTF();
ds.close();
What is the proper way of doing this?
I wrote a quick example on how to read .net's binaryWriter format in java here
excerpt from link:
/**
* Get string from binary stream. >So, if len < 0x7F, it is encoded on one
* byte as b0 = len >if len < 0x3FFF, is is encoded on 2 bytes as b0 = (len
* & 0x7F) | 0x80, b1 = len >> 7 >if len < 0x 1FFFFF, it is encoded on 3
* bytes as b0 = (len & 0x7F) | 0x80, b1 = ((len >> 7) & 0x7F) | 0x80, b2 =
* len >> 14 etc.
*
* #param is
* #return
* #throws IOException
*/
public static String getString(final InputStream is) throws IOException {
int val = getStringLength(is);
byte[] buffer = new byte[val];
if (is.read(buffer) < 0) {
throw new IOException("EOF");
}
return new String(buffer);
}
/**
* Binary files are encoded with a variable length prefix that tells you
* the size of the string. The prefix is encoded in a 7bit format where the
* 8th bit tells you if you should continue. If the 8th bit is set it means
* you need to read the next byte.
* #param bytes
* #return
*/
public static int getStringLength(final InputStream is) throws IOException {
int count = 0;
int shift = 0;
boolean more = true;
while (more) {
byte b = (byte) is.read();
count |= (b & 0x7F) << shift;
shift += 7;
if((b & 0x80) == 0) {
more = false;
}
}
return count;
}
As its name implies, BinaryWriter writes in binary format. .Net binary format to be precise, and as java is not a .Net language, it has no way of reading it. You have to use an interoperable format.
You can choose an existing format, like xml or json or any other interop format.
Or you can create your own, providing your data is simple enough to make it this way (it seems to be the case here). Just write a string to your file (using a StreamWriter for instance), provided you know your string's format. Then read your file from java as a string and parse it.
There is a very good explanation of the format used by BinaryWriter in this question Right Here it should be possible to read the data with a ByteArrayInputStream and write a simple translator.
So I have binary FRX files, from which I need to extract strings into Java.
I wrote this into my Java program like so:
FileInputStream ReadFRX = null ;
FileOutputStream TempCapt = null ;
try{
// refNum is hex number on end of VB form property converted to decimal, ex: $"frmResidency.frx":0134
int refNum = Integer.parseInt(line.substring(line.length() - 4, line.length()), 16);
// FRXtemp.txt is created, to temporarily write FRX captions onto to be read from.
PrintWriter writer = new PrintWriter("FRXtemp.txt", "UTF-8");
writer.close();
//opens corresponding FRX file to read into
ReadFRX = new FileInputStream("FRXFiles\\"+curFrmName + ".frx");
//aLittleEndian... must be used to match readInt() little-endianness
LittleEndianDataInputStream ActReadFRX = new LittleEndianDataInputStream(ReadFRX);
TempCapt = new FileOutputStream("FRXtemp.txt");
ActReadFRX.skipBytes(refNum);
int length = ActReadFRX.readInt();
int c;
for (c = 0; c < length; c++) {
// first read byte and check for EOF
TempCapt.write(ActReadFRX.read());
}
}
//If caption is not read properly (ie. possibly wrong bytes), EOF Exception will occur and designer will break
catch (EOFException e){
System.out.println("ERROR : FRX Caption property was mishandled");
break;
}
//Read data from FRXtemp.txt into string
String actCaption = "\"" + new Scanner(new File("FRXtemp.txt")).useDelimiter("\\A").next() + " \" ";
This works perfectly, however I think writing to a temporary file so that I can read off of it must be highly unnecessary.
Why I can't think of a more efficient method:
I feel like a much more practical approach would be to use a Byte[] Array, and then convert that to a string, however I must only have the bytes in which the string are stored. Research led me to believe that RandomAccessFile was then necessary so that I could set an offset from ReadInt to begin reading bytes , however RandomAccessFile assumes big-endian format, whereas I have little-endian format. I can obviously convert, however at that point my current solution seems just as viable.
My question is, is there an efficient way to convert a specific section of bytes corresponding to a 4-byte integer (from a binary file with little-endian format) into a string in Java?
I feel as though I must be overlooking something much more simple. Thanks :)
You can do this any number ways, however the simplest might be.
try (DataInputStream dis = new DataInputStream(new FileInputStream(file))) {
dis.skip(bytesToSkip);
int length = Integer.reverseBytes(dis.readInt());
byte[] bytes = new bytes[length];
dis.readFully(bytes);
return new String(bytes, "UTF-8");
}
The method you might have been looking for is in Integer
/**
* Returns the value obtained by reversing the order of the bytes in the
* two's complement representation of the specified {#code int} value.
*
* #param i the value whose bytes are to be reversed
* #return the value obtained by reversing the bytes in the specified
* {#code int} value.
* #since 1.5
*/
public static int reverseBytes(int i) {
return ((i >>> 24) ) |
((i >> 8) & 0xFF00) |
((i << 8) & 0xFF0000) |
((i << 24));
}
Something like this maybe?
long length = 0xff && mybytes[0]; length<<8;
length |= 0xff && mybytes[1]; length<<8;
length |= 0xff && mybytes[2]; length<<8;
length |= 0xff && mybytes[3]; length<<8;
You can use the inputStream that you have as the source and use a ByteBuffer to correct the endianess when creating the Strings as needed. This would be the most efficient way.