Develop Reference

How can I convert a .wav file to a float array ( edit the float array e.g. adding two signals) and back to a .wav file without whitenoise

I am trying to program an auralization via Ray-Tracing in processing. To edit a sample over the information from the Ray Tracer, i need to convert a .wav File (File-Format: PCM-signed,16bit,stereo,2 bytes/frame, little endian) to an Float Array.
I converted the audio via an audioInputStream and a DataInputStream, where I am loading the audio into an byte Array.
Then I convert the byte Array to a float array like this.
byte[] samples;
float[] audio_data = float(samples);
When I convert the float Array back to a .wav File, I'm getting the sound of the original Audio-File.
But when I'm adding another Float Array to the Original signal and convert it back to a. wav file via the method above(even if I'm adding the same signal), i get a white noise signal instead of the wanted signal (I can hear the original signal under the white noise modulated, but very very silent).
I read about this problem before, that there can be problems by the conversion from the float array to a byte array. That's because float is a 32bit datatype and byte (in java) is only 16 bits and somehow the bytes get mixed together wrong so the white noise is the result. In Processing there is a data type with signed 16bit integers (named: "short") but i can't modify the amplitude anymore, because therefore i need float values, which i can't convert to short.
I also tried to handle the overflow (amplitude) in the float array by modulating the signal from 16 bit values (-32768/32767) to values from -1/1 and back again after mixing (adding) the signals. The result gave me white noise. When i added more than 2 signals it gaves me nothing (nothing to hear).
The concrete Problem I want to solve is to add many signals (more than 1000 with a decent delay to create a kind of reverbation) in the form of float Arrays. Then I want to combine them to one Float Array that i want to save as an audio file without white noise.
I hope you guys can help me.

If you have true PCM data points, there should be no problem using simple addition. The only issue is that on rare occasions (assuming your audio is not too hot to begin with) the values will go out of range. This will tend create a harsh distortion, not white noise. The fact that you are getting white noise suggests to me that maybe you are not converting your PCM sums back to bytes correctly for the format that you are outputting.
Here is some code I use in AudioCue to convert PCM back to bytes. The format is assumed to be 16-bit, 44100 fps, stereo, little-endian. I'm working with PCM as normalized floats. This algorithm does the conversion for a buffer's worth of data at a time.
for (int i = 0, n = buffer.length; i < n; i++)
{
buffer[i] *= 32767;
audioBytes[i*2] = (byte) buffer[i];
audioBytes[i*2 + 1] = (byte)((int)buffer[i] >> 8 );
}
Sometimes, a function like Math.min(Math.max(audioval, -1), 1) or Math.min(Math.max(audioval, -32767), 32767) is used to keep the values in range. More sophisticated limiters or compressor algorithms will scale the volume to fit. But still, if this is not handled, the result should be distortion, not white noise.
If the error is happening at another stage, we will need to see more of your code.
All this said, I wish you luck with the 1000-point echo array reverb. I hadn't heard of this approach working. Maybe there are processors that can handle the computational load now? (Are you trying to do this in real time?) My only success with coding real-time reverberation has been to use the Schroeder method, plugging the structure and values from the CCMRA Freeberb, working off of code from Craig Lindley's now ancient (copyright 2001) book "Digital Audio with Java". Most of that book deals with obsolete GUI code (pre-Swing!), but the code he gives for AllPass and Comb filters is still valid.
I recall when I was working on this that I tracked down references a better reverb to try and code, but I would have to do some real digging to try and find my notes. I was feeling over my head at the time, as the algorithm was presented via block diagrams not coding details or even pseudo-code. Would like to work on this again though and get a better reverb than the Shroeder-type to work. The Schoeder was passable for sounds that were not too percussive.
Getting a solution for real-time ray tracing would be a valuable accomplishment. Many applications in AR/VR and games.

Cap'n Proto - Finding Message Size in Java

I am using a TCP Client/Server to send Cap'n Proto messages from C++ to Java.
Sometimes the receiving buffer may be overfilled or underfilled and to handle these cases we need to know the message size.
When I check the size of the buffer in Java I get 208 bytes, however calling
MyModel.MyMessage.STRUCT_SIZE.total()
returns 4 (not sure what unit of measure is being used here).
I notice that 4 divides into 208, 52 times. But I don't know of a significant conversion factor using 52.
How do I check the message size in Java?

MyMessage.STRUCT_SIZE represents the constant size of that struct itself (measured in 8-byte words), but if the struct contains non-trivial fields (like Text, Data, List, or other structs) then those take up space too, and the amount of space they take is not constant (e.g. Text will take space according to how long the string is).
Generally you should try to let Cap'n Proto directly write to / read from the appropriate ByteChannels, so that you don't have to keep track of sizes yourself. However, if you really must compute the size of a message ahead of time, you could do so with something like:
ByteBuffer[] segments = message.getSegmentsForOutput();
int total = (segments.length / 2 + 1) * 8; // segment table
for (ByteBuffer segment: segments) {
total += segment.remaining();
}
// now `total` is the total number of bytes that will be
// written when the message is serialized.
On the C++ size, you can use capnp::computeSerializedSizeInWords() from serialize.h (and multiply by 8).
But again, you really should structure your code to avoid this, by using the methods of org.capnproto.Serialize with streaming I/O.

Parsing a TCP packet

I'm having some trouble to parse a TCP packet from a socket...
In my protocol, my messages are like this:
'A''B''C''D''E'.........0x2300
'A''B''C''D''E' --> start message pattern
0x2300 --> two bytes end message
But due to the Nagle's algorithm, sometimes my messages are concatenated like:
'A''B''C''D''E'.........0x2300'A''B''C''D''E'.........0x2300'A''B''C''D''E'.........0x2300
I already tried to setNoDelay() to true but the problem persists.
I have the message in a byte[].
How could I split my messages to be parsed individually?
PS: For now I am able to get the first message but the others are lost...

Just loop through you received data and check for end-markers. When found set a start index to the next package and continue searching. Something like this:
int packageStart = 0;
for(int i = 0; i < data.length - 1; i++) {
if(data[i] == 0x23 && data[i + 1] == 0x00) {
// Found end of package
i++;
processPackage(data, packageStart, i);
packageStart = i;
}
// At this point: from packageStart till data.length are unprocessed bytes...
As noted, there might be some left over data (if data did not end with the end-marker). You might want to keep it, so you can prepend it to the next batch of received data. And thus preventing data-loss due to chopped up TCP/IP packages.

You have to think of it as parsing a continuous stream of bytes. Your code needs to identify the start and end of a message.
Due to the way packets get sent, you may have a complete message, multiple messages, a partial message, etc. You code needs to identify when a message has begun and keep reading until it has found the end of a message or in some instance, when you've read more bytes than your max message size and you need to resync.
I've seen some comm managers drop and reestablish the connection (start over) and others throw away data until they can get back in sync. Then you get into the fun of whether you need guaranteed delivery and retransmission.
The best protocols are the simple ones. Create a message header which contains say an SOH byte, a two byte message length (or whatever is appropriate), a 2 byte message type and 1 byte message subtype. You can also end the message with any number of bytes. Look at an ASCII chart, there's a number of Hex bytes 00-1F that are pretty standard since the terminal days.
No point in reinventing the wheel here. Makes it easier, because you know how long this message should be instead of looking for patterns in the data.

It sounds like you need to treat it like a Byte Stream and buffer the packets until you see your EOF code 0x2300.

Parse binary response

I am trying to build a sample application which will show a proof of concept for synchronizing the time with an RFC 868 compliant time server.
So far, using the Java Socket API, I am able to connect and query the server and do get the response from the server, but it is not in human readable format.
The response I get is: �)6 I think the response is coming in binary format (not sure though). RFC 868 says that Send the time as a 32 bit binary number.
My questions are:
How do I parse this response?
Apart from this approach of mine, I'd like to know if there is any other recommended approach which I should take to achieve this.
Thanks in advance.

1) How do I parse this response?
Check out the source code of TimeTCPClient from Apache Commons Net library:
public long getTime() throws IOException {
DataInputStream input;
input = new DataInputStream(_input_);
return (input.readInt() & 0xffffffffL);
}
public Date getDate() throws IOException {
return new Date((getTime() - SECONDS_1900_TO_1970)*1000L);
}
2) Apart from this approach of mine, I'd like to know if there is any other recommended approach which I should take to achieve this.
Use Apache Commons Net Library, check out the API of TimeTCPClient.
Apache Commons Net home page, hope this helps.

As stated in the RFC this is the seconds since 1900-01-01T00:00:00. For Java convert it to a Long,change the base date to 1970-01-01T00:00:00, and multiply by 1000 to get the date. Then you can create a new Date using this value.
Wrap your socket input stream to a DataInputStream and read an into rfsOffset (I used a constant). Then you can do something like:
int rfcOffset = -752253627; // Fri Apr 06 11:00:32 EDT 2012
// Current offsets will be negative convert to long positive value
long offsetSecs = rfcOffset + 4294967296L;
System.out.println(offsetSecs);
// Adjust time base from 1900 to 1970 and convert to millis
long offsetMillis = ( offsetSecs - 2208988800L)* 1000L;
System.out.println(offsetMillis);
Date rfcDate = new Date(offsetMillis);
System.out.println(rfcDate.toString());
Note: this only works until 2036 and time will be off by some number of milliseconds.
EDIT: RFC 868 is an old protocol and is no longer considered a good time source for synchronization. A good time source will us NTP and will return the correct second. It may be off a few milliseconds, but is normally accurate withing 10 milliseconds. Many hardware clocks drift significantly, and I have seen significant drift from systems with inaccurate clocks (even with NTP running(). NTP will correct a drifting clock, but needs a few minutes to determine the required shift.
EDIT2: While RFC 868 is old, it may be good enough to set the time on a cell phone to the nearest second without requiring a background process. This shouldn't be necessary if your cell phone can sync to a signal sent by your provider.

How to parse bit fields from a byte array in Java?

I've been given the arduous task of parsing some incoming UDP packets from a source into an appropriate Java representation. The kicker is the data held within the packets are not byte aligned. In an effort to make the protocol as tight as possible, there are a number of bit fields indicating the presence or absence of data fields.
For example, at bit index 34 you may find a 24 bit field that should be converted to a float. Bit index 110 may be a flag indicating that the next 3 fields are each 5 and 6 bit values containing the hour, minute, and second of the day. (These are just made up by me but are representative of what the spec says). These packets are probably a few hundred bits long.
The spec is not likely to change, but it's completely possible that I'll be asked to decode other similar packet formats.
I can certainly bit shift and mask as needed, but I'm worried about ending up in bit shift hell and then drowning in it when more packet formats are discovered.
I'd love to hear any suggestions on best practices or Java libraries that may make the task more manageable.

Decoding QR codes is much the same exercise in reading a couple bits at a time, completely unaligned. Here's what I wrote to handle it -- maybe you can just reuse this.
http://code.google.com/p/zxing/source/browse/trunk/core/src/com/google/zxing/common/BitSource.java

for such cases I have developed JBBP library which is accessible in Maven central
for instance parsing of file to bits and printing of parsed values looks like
public static final void main(final String ... args) throws Exception {
try (InputStream inStream = ClassLoader.getSystemClassLoader().getResourceAsStream("somefile.txt")) {
class Bits { #Bin(type = BinType.BIT_ARRAY) byte [] bits; }
for(final byte b : JBBPParser.prepare("bit [_] bits;",JBBPBitOrder.MSB0).parse(inStream).mapTo(Bits.class).bits)System.out.print(b != 0 ? "1" : "0");
}
}