I am actually develop an app (spectrum analyzer) on smartphone. This app can communicate by bluetooth with a raspberry pi 3 and scan frequency from my RTLSDR-2832 (plug on RPI3).
I wrote a script to get samples from my RTLSDR-2832 and I send it on my app by bluetooth :
from pylab import*
from rtlsdr import*
from bluetooth import*
import sys
#configure device
sdr= RtlSdr()
sdr.sample_rate=double(sys.argv[3])
sdr.gain=double(sys.argv[2])
sdr.center_freq=double(sys.argv[1])
NFFT=50
#Bluetooth connection
server_sock=BluetoothSocket(RFCOMM)
server_sock.bind(("",PORT_ANY))
server_sock.listen(1)
port=server_sock.getsockname()[1]
uuid="94f39d29-7d6d-437d-973b-fba39e49d4ee"
client_sock,client_info=server_sock.accept()
while(1):
samples=sdr.read_samples(256*1024)
result=psd(samples,NFFT, Fs=sdr.sample_rate/1e6, Fc=sdr.center_freq*1e6/1e6)
tab_freq=(result[1]/1e6)
value_freq=str(tab_freq)[1:-1]
value_list=[format(float(v), ".10f") for v in value_freq.split()]
value_freq2= "\n".join(value_list)
tab_pxx=result[0]
value_pxx=str(tab_pxx)[1:-1]
client_sock.send(value_freq2+'\n'+'\n'.join(value_pxx.split()))
So with this script, I can get data from my RTLSDR2838 and send data. I send by bluetooth all data freq in first then, pxx value (in db/Hz).
I get all data on my app with that :
private class ThreadConnected extends Thread {
private final BluetoothSocket connectedBluetoothSocket;
private final InputStream connectedInputStream;
private final OutputStream connectedOutputStream;
boolean running;
public ThreadConnected(BluetoothSocket socket) {
connectedBluetoothSocket = socket;
InputStream in = null;
OutputStream out = null;
running = true;
try {
in = socket.getInputStream();
out = socket.getOutputStream();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
connectedInputStream = in;
connectedOutputStream = out;
}
#Override
public void run() {
byte[] buffer = new byte[1048576]; // 20 bits
int bytes;
String strRx = "";
while (running) {
try {
bytes = connectedInputStream.read(buffer);
final String strReceived_freq = new String(buffer,0, 15*nb_points);
final String strReceived_pxx = new String(buffer,15*nb_points, bytes);
//final int samples_sdr=new Integer(buffer,0,bytes);
final String strByteCnt = String.valueOf(bytes) + " bytes received.\n";
runOnUiThread(new Runnable(){
#Override
public void run() {
Pxx_value.setText(strReceived_pxx+"\n"); // get data PXX
freq_value.setText(strReceived_freq+"\n"); // get data freq
// plot value
/* for (int i=0; i<nb_points; i++)
{
freq[i]=Double.parseDouble(strReceived_freq);
pxx[i]=Double.parseDouble(strReceived_pxx);
series.appendData(new DataPoint(freq[i],pxx[i]), true,500);
}*/
}});
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
cancel();
final String msgConnectionLost = "Connection lost:\n" + e.getMessage();
runOnUiThread(new Runnable(){
#Override
public void run() {
}});
}
}
}
public void write(byte[] buffer) {
try {
connectedOutputStream.write(buffer);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void cancel() {
running = false;
try {
connectedBluetoothSocket.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The variable nb_points is fix to 50 like NFFT on my script. But when I did that I don't get my 50 values. If I try to fix nb_points and NFFT at 30 or less it works but not after...
I thinked at early maybe the problem is from the buffer but no apparently...
Did someone have any idea to fix this ?
Thanks for your help !
EDIT : It works until 33 exactly, after I don't know if it's about my buffer or not, but for each case of my tab, I get 2 values in sames times (my frequence, and pxx value) as the tab is writing again
I got a very strange behaviour by reading bytes from the input stream of a socket.
In my project clients does requests to a service. For each request a new connection will be established.
At first the bytes are send that tells the service what kind of request will follow.
Then the request itself is send.
The service receives the bytes and proceeds the request. This does work for at least 95% of all request. For the remaining 5% there is a strange behaviour that i can not figure out.
The bytes are not all the bytes that got sended. But the most strange matter on this topic is that the missing bytes are not at the start or at the end of the stream. They are spread through the entire stream.
Sadly i can not provide the full code here cause it is work related. But i can provide the test code that shows the issue itself.
To figure out what is going on i wrote 2 classes. One derives from java.net.Socket and the other one from java.net.ServerSocket.
Here the code:
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.util.ArrayList;
import java.util.List;
public class DebugSocket extends Socket
{
private class InputStreamWrapper extends InputStream
{
private int
availables,
closes,
marksupporteds,
resets;
private List<Integer>
marks = new ArrayList<Integer>(),
reads = new ArrayList<Integer>();
private List<Long>
skips = new ArrayList<Long>();
#Override
public int available() throws IOException
{
availables++;
return DebugSocket.this.origininput.available();
}
#Override
public void close() throws IOException
{
closes++;
DebugSocket.this.origininput.close();
}
#Override
public synchronized void mark(int readlimit)
{
marks.add(readlimit);
DebugSocket.this.origininput.mark(readlimit);
}
#Override
public boolean markSupported()
{
marksupporteds++;
return DebugSocket.this.origininput.markSupported();
}
#Override
public synchronized void reset() throws IOException
{
resets++;
DebugSocket.this.origininput.reset();
}
#Override
public int read() throws IOException
{
int read = DebugSocket.this.origininput.read();
reads.add(read);
if ( read != -1 )
{
DebugSocket.this.inputdebugbuffer.write(read);
}
return read;
}
#Override
public int read(byte[] b) throws IOException
{
int read = DebugSocket.this.origininput.read(b);
DebugSocket.this.inputdebugbuffer.write(b, 0, read);
return read;
}
#Override
public int read(byte[] b, int off, int len) throws IOException
{
int read = DebugSocket.this.origininput.read(b, off, len);
DebugSocket.this.inputdebugbuffer.write(b, off, read);
return read;
}
#Override
public long skip(long n) throws IOException
{
long skipped = DebugSocket.this.origininput.skip(n);
skips.add(skipped);
return skipped;
}
}
private class OutputStreamWrapper extends OutputStream
{
private int
flushes,
closes;
#Override
public void close() throws IOException
{
closes++;
DebugSocket.this.originoutput.close();
}
#Override
public void flush() throws IOException
{
flushes++;
DebugSocket.this.originoutput.flush();
}
#Override
public void write(int b) throws IOException
{
DebugSocket.this.outputdebugbuffer.write(b);
DebugSocket.this.originoutput.write(b);
DebugSocket.this.originoutput.flush();
}
#Override
public void write(byte[] b) throws IOException
{
DebugSocket.this.outputdebugbuffer.write(b);
DebugSocket.this.originoutput.write(b);
DebugSocket.this.originoutput.flush();
}
#Override
public void write(byte[] b, int off, int len) throws IOException
{
DebugSocket.this.outputdebugbuffer.write(b, off, len);
DebugSocket.this.originoutput.write(b, off, len);
DebugSocket.this.originoutput.flush();
}
}
private static final Object
staticsynch = new Object();
private static long
idcounter = 0;
private final long
id;
private final ByteArrayOutputStream
inputdebugbuffer,
outputdebugbuffer;
private final InputStream
inputwrapper;
private final OutputStream
outputwrapper;
private InputStream
origininput;
private OutputStream
originoutput;
public InputStream getInputStream() throws IOException
{
if ( origininput == null )
{
synchronized ( inputdebugbuffer )
{
if ( origininput == null )
{
origininput = super.getInputStream();
}
}
}
return inputwrapper;
}
public OutputStream getOutputStream() throws IOException
{
if ( originoutput == null )
{
synchronized ( outputdebugbuffer )
{
if ( originoutput == null )
{
originoutput = super.getOutputStream();
}
}
}
return outputwrapper;
}
public DebugSocket()
{
id = getNextId();
inputwrapper = new InputStreamWrapper();
outputwrapper = new OutputStreamWrapper();
inputdebugbuffer = new ByteArrayOutputStream();
outputdebugbuffer = new ByteArrayOutputStream();
}
private static long getNextId()
{
synchronized ( staticsynch )
{
return ++idcounter;
}
}
}
import java.io.IOException;
import java.net.ServerSocket;
public class DebugServerSocket extends ServerSocket
{
public DebugServerSocket() throws IOException
{
super();
}
public DebugSocket accept() throws IOException
{
DebugSocket s = new DebugSocket();
implAccept(s);
return s;
}
}
The class DebugSocket takes notification of each interaction with the InputStream as well as OutputStream
Now when the issue occurs i always can see that bytes are missing.
Here an example:
The client send 1758 bytes. I got the 23 top bytes from the member outputdebugbuffer in the DebugSocket.
Bytes: 0,0,0,0,0,0,0,2,0,0,6,-46,31,-117,8,0,0,0,0,0,0,0,-83
The server received 227 Bytes. For debug issues i always do read the input stream till i get a -1, so that all bytes got proceeded. Now the 16 leading bytes on serverside that i got from the member inputdebugbuffer in the DebugSocket.
Bytes: 0,0,0,6,-46,31,-117,8,0,0,0,0,0,0,0,-83
As shown there are 7 bytes missing. the first 8 bytes are a long value this one i changed to a byte value for debugging. So i figured that the first byte is always correct.
If it were a failure in the code no request would be proceeded but as i said before this happens only to 5% of all connections at best.
Got anyone an idea whats going on here?
I also used the DataInputStream and DataOutputStream to send the data. I always flush after each write operation as you can see in the OutputStreamWrapper of the DebugSocket.
Do i miss something here?
If some other code is required i will try to post it.
P.S. The service is multi threaded and processes 100 request parallel. Also the clients are multi threaded and do 20 requests parallel. As said each request uses its one connection and closes this one right after the request got proceeded.
I hope someone got an idea on this matter.
Edit:
There is no main method to show that does anything like asked in the comments but here the the code blocks of the client and the server that are used.
Client: (Run parallel in 20 threads)
public void sendRequest(long _requesttype, byte[] _bytes)
{
Socket socket = null;
DataInputStream input = null;
DataOutputStream output = null;
InputStream sinput = null;
OutputStream soutput = null;
try
{
socket = new DebugSocket();
socket.connect(serveraddress);
sinput = socket.getInputStream();
soutput = socket.getOutputStream();
input = new DataInputStream(sinput);
output = new DataOutputStream(soutput);
output.writeLong(_requesttype);
output.flush();
soutput.flush();
output.write(_bytes);
output.flush();
soutput.flush();
// wait for notification byte that service had received all data.
input.readByte();
}
catch (IOException ex)
{
LogHelper.log(ex);
}
catch (Error err)
{
throw err;
}
finally
{
output.flush();
soutput.flush();
input.close();
output.close();
finishSocket(socket);
}
}
Server: (Run in a thread for each request. Up to 100 threads)
public void proceedRequest(DebugSocket _socket)
{
DataInputStream input = null;
DataOutputStream output = null;
InputStream sinput = null;
OutputStream soutput = null;
try
{
sinput = _socket.getInputStream();
soutput = _socket.getOutputStream();
input = new DataInputStream(sinput);
output = new DataOutputStream(soutput);
RequestHelper.proceed(input.readLong(), input, output);
// send notification byte to the client.
output.writeByte(1);
output.flush();
soutput.flush();
}
catch (IOException ex)
{
LogHelper.log(ex);
}
catch (Error err)
{
throw err;
}
finally
{
output.flush();
soutput.flush();
input.close();
output.close();
}
}
In the server code the readLong() already fails cause of the missing bytes.
Ok im done with all possible ways to locate the cause. From my experience with socket programming and parallel processing i can say that there is no bug in the code itself. Sniffers as well tell me that. Something on my machine is messing with the transmission.
I deactivated all i could think of (firewall/antivir/malware scanner) but no effect.
Got someone an idea what else could mess with tcp packages?
Edit:
Ok i got it. AVG 2014 is messing. Jetzt deactivating the components did not work. In Options->Settings there is a menu point were you can deactivate the AVG-Protection.
Got someone knowledge on this topic?
My guess is there is a bug somewhere else in the code. I copied the DebugSocket class from the question and created a MCVE (see below). It works fine, I was unable to reproduce the "server cannot read long-value" problem. Try modifying the code below to include more of the your own code until you can reproduce the problem, that should give you an idea where to look for the underlying cause.
import java.io.*;
import java.net.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
public class TestDebugSocket implements Runnable, Closeable {
public static void main(String[] args) {
TestDebugSocket m = new TestDebugSocket();
try {
m.run();
} catch (Exception e) {
e.printStackTrace();
} finally {
m.close();
}
}
final int clients = 20;
final boolean useDebugSocket = true;
final byte[] someBytes = new byte[1758];
final ThreadPoolExecutor tp = (ThreadPoolExecutor) Executors.newCachedThreadPool();
final AtomicLong clientId = new AtomicLong();
final ConcurrentLinkedQueue<Closeable> closeables = new ConcurrentLinkedQueue<Closeable>();
final long maxWait = 5_000L;
final CountDownLatch serversReady = new CountDownLatch(clients);
final CountDownLatch clientsDone = new CountDownLatch(clients);
ServerSocket ss;
int port;
#Override public void run() {
try {
ss = useDebugSocket ? new DebugServerSocket() : new ServerSocket();
ss.bind(null);
port = ss.getLocalPort();
tp.execute(new SocketAccept());
for (int i = 0; i < clients; i++) {
ClientSideSocket css = new ClientSideSocket();
closeables.add(css);
tp.execute(css);
}
if (!clientsDone.await(maxWait, TimeUnit.MILLISECONDS)) {
System.out.println("CLIENTS DID NOT FINISH");
} else {
System.out.println("Finished");
}
} catch (Exception e) {
e.printStackTrace();
} finally {
close();
}
}
#Override public void close() {
try { if (ss != null) ss.close(); } catch (Exception ignored) {}
Closeable c = null;
while ((c = closeables.poll()) != null) {
try { c.close(); } catch (Exception ignored) {}
}
tp.shutdownNow();
}
class DebugServerSocket extends ServerSocket {
public DebugServerSocket() throws IOException {
super();
}
#Override public DebugSocket accept() throws IOException {
DebugSocket s = new DebugSocket();
implAccept(s);
return s;
}
}
class SocketAccept implements Runnable {
#Override public void run() {
try {
for (int i = 0; i < clients; i++) {
SeverSideSocket sss = new SeverSideSocket(ss.accept());
closeables.add(sss);
tp.execute(sss);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
class SeverSideSocket implements Runnable, Closeable {
Socket s;
public SeverSideSocket(Socket s) {
this.s = s;
}
#Override public void run() {
Long l = -1L;
byte[] received = new byte[someBytes.length];
try {
DataInputStream in = new DataInputStream(s.getInputStream());
DataOutputStream out = new DataOutputStream(s.getOutputStream());
serversReady.countDown();
if (!serversReady.await(maxWait, TimeUnit.MILLISECONDS)) {
System.out.println("CLIENTS DID NOT CONNECT ON TIME TO SERVER");
}
l = in.readLong();
in.readFully(received);
out.writeByte(1);
out.flush();
} catch (Exception e) {
e.printStackTrace();
} finally {
// write to console at end to prevent synchronized socket I/O
System.out.println("received long: " + l);
close();
}
}
#Override public void close() {
TestDebugSocket.close(s);
s = null;
}
}
class ClientSideSocket implements Runnable, Closeable {
Socket s;
#SuppressWarnings("resource")
#Override public void run() {
Long l = -1L;
Byte b = -1;
try {
s = useDebugSocket ? new DebugSocket() : new Socket();
s.connect(new InetSocketAddress(port));
DataInputStream in = new DataInputStream(s.getInputStream());
DataOutputStream out = new DataOutputStream(s.getOutputStream());
l = clientId.incrementAndGet();
out.writeLong(l);
out.write(someBytes);
out.flush();
b = in.readByte();
} catch (Exception e) {
e.printStackTrace();
} finally {
System.out.println("long send: " + l + ", result: " + b);
close();
clientsDone.countDown();
}
}
#Override public void close() {
TestDebugSocket.close(s);
s = null;
}
}
static void close(Socket s) {
try { if (s != null) s.close(); } catch (Exception ignored) {}
}
}
I would like to test the connection between a client and a server in a ScheduledExecutorService every x ms while processing received data from the distant host.
So I did something like this:
public class MyClass {
private final ScheduledExecutorService _timer = Executors.newScheduledThreadPool(1);
private Socket _connection;
public void connectToDistantHost() {
try {
_connection = new Socket();
_connection.connect(_adresseServeur);
new Thread(new Runnable() {
#Override
public void run() {
try {
//let another object know the connection is ok
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
} catch (IOException e) {
e.printStackTrace();
}
_timer.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
testConnection();
}
}, 0, 200, TimeUnit.MILLISECONDS);
}
private void testConnection() {
//would like to peek on the socket's inputstream to know if something's wrong
}
private void myProcessing() {
while (true) {
...
//read what's inside stream
//process it in a thread
}
}
}
So, if I .read() on the Socket's inputstream it'll screw myProcessing(). I thought about wraping the inputstream in a BufferedReader and mark the buffer position before I read and then reset the position, but as the testing and the processing are in two differents thread it won't work anyway.
How can I do that? Knowing that I did it in c# without much problem:
class TraitementEnvoiClient {
...
private void testConnection(Object obj, ElapsedEventArgs args) {
_connectionIsOk = _connexionAuServeur.IsConnected();
if (!_connectionIsOk) {
tryToReconnect();
}
}
}
public static class ExtensionTcpClient {
//Credit to ElFenix: http://social.msdn.microsoft.com/Forums/en-US/c857cad5-2eb6-4b6c-b0b5-7f4ce320c5cd/c-how-to-determine-if-a-tcpclient-has-been-disconnected?forum=netfxnetcom
public static bool IsConnected(this TcpClient client) {
// Detect if client disconnected
try {
if (client.Client.Poll(0, SelectMode.SelectRead)) {
byte[] buff = new byte[1];
if (client.Client.Receive(buff, SocketFlags.Peek) == 0) {
// Client disconnected
return false;
}
}
} catch (SocketException se) {
return false;
}
return true;
}
}
Thank you
Edit: I would like to make something like that:
private static boolean isConnected(Socket client) {
try {
InputStream is = client.getInputStream();
if(is.peek() == -1) return false;
OutputStream os = client.getOutputStream();
os.write(new byte[]{}); //if it fails a IOException will trigger
} catch(SocketException se) {
return false;
} catch(IOException ioe) {
return false;
}
return true;
}
The testing is redundant. The read or write operations will return -1 if the other end closes or disconnects before or during the operation. There is no point in "testing" the connection first because it may subsequently fail during your IO operation.
See also the other answer mentioned in the comments.
I'm trying to pipe some ByteBuffer in a thread (IO1) to another (IO2).
http://tutorials.jenkov.com/java-nio/pipe.html
private int bufferSize;
private boolean isRecording;
private Thread IO1;
private Thread IO2;
private ByteBuffer byteBuffer1;
private ByteBuffer byteBuffer2;
private Pipe pipe;
private Pipe.SinkChannel skChannel;
private Pipe.SourceChannel sourceChannel;
byteBuffer1 = ByteBuffer.allocateDirect(bufferSize);
byteBuffer2 = ByteBuffer.allocateDirect(bufferSize);
//An instance of Pipe is created
try
{
pipe = Pipe.open();
skChannel = pipe.sink();
sourceChannel = pipe.source();
IO1.start();
IO2.start();
}
catch (IOException e)
{
e.printStackTrace();
}
--
IO1 = new Thread(new Runnable()
{
public void run()
{
isRecording = true;
recorder.startRecording();
try
{
int read;
while (isRecording)
{
// byteBuffer1.clear();
read = recorder.read(byteBuffer1, bufferSize);
if (AudioRecord.ERROR_INVALID_OPERATION != read)
{
skChannel.write(byteBuffer1);
Log.v("========IO1 ", String.valueOf(read));
//This triggered almost 20 times/second
}
}
recorder.stop();
}
catch (Exception e)
{
e.printStackTrace();
}
}
});
skChannel.write(byteBuffer1); and
Log.v("========IO1 ", String.valueOf(read));
is triggered almost 20 times/second, and this is the expected behavior, so far so good.
IO2 = new Thread(new Runnable()
{
public void run()
{
try
{
int read;
while ( (read =sourceChannel.read(byteBuffer2)) >0)
{
Log.v("========IO2 ", String.valueOf(read));
//this triggered only once
// To do Codec etc.
//............
}
}
catch (IOException e)
{
e.printStackTrace();
}
Log.v("========IO2 ", "END!!!!!"); //never triggered(this is fine)
}
});
However, Log.v("========IO2 ", String.valueOf(read)); is triggered only once, and I don't know why.
Can someone tell me how can I obtain the update of Thread IO1 in IO2?
Thanks.
You need to flip() the buffer before writing, and compact() it afterwards.
BUT: In a word, don't. Pipes between threads are basically pointless. Use a queue, or have the receiving thread read the sending thread's input directly.
If you must do this, the basic NIO copy loop goes like this:
while (in.read(buffer) > 0 || buffer.position() > 0) // or whatever your read API needs
{
buffer.flip();
out.write(buffer);
buffer.compact();
}
Under what circumstances read end can be dead in couple PipedOutputStream and PipedInputStream? I am not closing any pipes.
I encountered java.io.IOException: Read end dead in my code and found out the cause. Posting an example code below. You will get an "Read end dead" exception if you run the code. If you take a close look, the consumer thread reads "hello" from the stream and terminates; meanwhile the producer sleeps for 2 seconds and tries to write " world" but fails. A related problem explained here: http://techtavern.wordpress.com/2008/07/16/whats-this-ioexception-write-end-dead/
class ReadEnd {
public static void main(String[] args) {
final PipedInputStream in = new PipedInputStream();
new Thread(new Runnable() { //consumer
#Override
public void run() {
try {
byte[] tmp = new byte[1024];
while (in.available() > 0) { // only once...
int i = in.read(tmp, 0, 1024);
if (i < 0)
break;
System.out.print(new String(tmp, 0, i));
}
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
}).start();
PipedOutputStream out = null;
try {
out = new PipedOutputStream(in);
out.write("hello".getBytes());
Thread.sleep(2 * 1000);
out.write(" world".getBytes()); //Exception thrown here
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
}
}
}