Develop Reference

How to use Scanner to read data received continously over a period of time in android?

I am new in dealing with android network requests .
I need to make a request to an api to access some data in Json format . I am able to receive data and save it in database with no problem but when I try to make multiple requests , I get errors with Scanner class ("api here supports multiple network requests").
Here is how I am connecting to the api
connection = (HttpURLConnection) se.openConnection();
if (connection.getResponseCode() == (HttpURLConnection.HTTP_NOT_FOUND)) {
getActivity().runOnUiThread(new Runnable() {
#Override
public void run() {
progressbar.setVisibility(View.GONE);
Toast.makeText(getContext(), Constants.NO_MEANING_FOUND, Toast.LENGTH_LONG).show();
}
});
} else {
Log.d("enteredtt","hellooo");
inputStream = connection.getInputStream();
sc = new Scanner(inputStream);
sc.useDelimiter("\\A");
Json_raw = sc.next(); // the error occures here
There are two exceptions thrown which are
java.lang.IllegalStateException: expected null but was com.android.okhttp.internal.http.Http1xStream#4e45274
and java.util.NoSuchElementException
at java.util.Scanner.throwFor(Scanner.java:879)
at java.util.Scanner.next(Scanner.java:1394)
The above code works well with one or two requests at a time. I also used singlethreadpool executors to make requests but the result is same .Is there something I need to follow while make severeal network requests or is it something to do with storing buffer data ? I will provide more code if needed .

Why is DataInputStream.readFully(...) hanging?

I'm having a difficult time understanding what is going wrong in my application. Below shows code that successfully executes on first pass (a prior HttpResponse, and then on its second call, hangs at the dataInputStream.readFully(...) line.
doSomething(HttpResponse response) {
HttpEntity responseBody = response.getEntity();
long len = responseBody.getContentLength();
byte[] payload = new byte[(int)len]; // <-- I've confirmed this is the correct length
DataInputStream d = null;
try {
InputStream bais = responseBody.getContent();
d = new DataInputStream(bais);
d.readFully(payload); // <-- *** HANGS HERE! ***
EntityUtils.consume(responseBody);
...
} finally {
if (d != null) {
IOUtils.closeQuietly(d);
}
}
}
After blocking/hanging for 10+ seconds, the application times out and the calling thread is torn down.
I've inspected the response object, and confirmed its Content-Length to be what is expected.
Nothing jumped out at me reading the DataInputStream Javadoc.
Going through our commit history, I noticed that the IOUtils.closeQuietly(...) call was introduced. [Trust me here:] It's been hard to track our integration system's initial failure of this application, so I cannot confirm whether this finally block introduced any unintended behavior. From what I can tell, calling closeQuietly() there is the recommended approach.
Update: For clarity, this code is part of an application performing HTTP RANGED GET requests. E.g., first response is pulling down N bytes, starting at byte 0. The second call GETs the next N bytes. The file is large, so there are more than two necessary RANGED GETs.

How do I Execute Java from Java?

I have this DownloadFile.java and downloads the file as it should:
import java.io.*;
import java.net.URL;
public class DownloadFile {
public static void main(String[] args) throws IOException {
String fileName = "setup.exe";
// The file that will be saved on your computer
URL link = new URL("http://onlinebackup.elgiganten.se/software/elgiganten/setup.exe");
// The file that you want to download
// Code to download
InputStream in = new BufferedInputStream(link.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n = 0;
while (-1 != (n = in.read(buf))) {
out.write(buf, 0, n);
}
out.close();
in.close();
byte[] response = out.toByteArray();
FileOutputStream fos = new FileOutputStream(fileName);
fos.write(response);
fos.close();
// End download code
System.out.println("Finished");
}
}
I want to execute this from a mouse event in Gui.java.
private void jLabel17MouseClicked(java.awt.event.MouseEvent evt){
}
How do I do this?

Your current method is a static method, which is fine, but all the data that it extracts is held tightly within the main method, preventing other classes from using it, but fortunately this can be corrected.
My suggestion:
re-write your DownloadFile code so that it is does not simply a static main method, but rather a method that can be called by other classes easily, and that returns the data from the file of interest. This way outside classes can call the method and then receive the data that the method extracted.
Give it a String parameter that will allow the calling code to pass in the URL address.
Give it a File parameter for the file that it should write data to.
Consider having it return data (a byte array?), if this data will be needed by the calling program.
Or if it does not need to return data, perhaps it could return boolean to indicate if the download was successful or not.
Make sure that your method throws all exceptions (such as IO and URL excptions) that it needs to throw.
Also, if this is to be called by a Swing GUI, be sure to call this type of code in a background thread, such as in a SwingWorker, so that this code does not tie up the Swing event thread, rendering your GUI frozen for a time.

Apache Wicket: File download that would not lock the page

I want to create a link that would initiate a file download which would be asynchronous to the page itself, i.e. I want the page not to be locked during the file download. Should I make it be initiated outside wicket? Or is there something inside wicket that would let me set up a resource stream which would bypass the page locks?
Things I tried:
DownloadLink - locks the page, as stated in its doc. This was my starting point.
ResourceLink - did not state the locking explicitly in the doc, so I tried this, but it also locked the page.
At this point I've investigated the code of both links a bit and noticed they both schedule the download via ResourceStreamRequestHandler. Expecting that his kind of behavior could be just handler-specific I've attempted to schedule a custom handler I've written:
private void sendFile(final File file) throws IOException {
IRequestHandler fileDownloadHandler = new IRequestHandler() {
#Override
public void respond(IRequestCycle requestCycle) {
WebResponse response = (WebResponse) requestCycle.getResponse();
OutputStream outStream = response.getOutputStream();
response.setContentType("audio/x-wav");
response.setContentLength((int)file.length());
String fileName = "Somethingsomething.wav";
// sets HTTP header
response.setHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
byte[] byteBuffer = new byte[1024];
DataInputStream in = null;
try {
in = new DataInputStream(new FileInputStream(file));
int length = 0;
// reads the file's bytes and writes them to the response stream
while ((in != null) && ((length = in.read(byteBuffer)) != -1))
{
outStream.write(byteBuffer,0,length);
}
in.close();
outStream.close();
} catch (IOException e) {
throw new PortalError("IOException trying to write the response", e);
}
}
#Override
public void detach(IRequestCycle requestCycle) {
}
};
getRequestCycle().scheduleRequestHandlerAfterCurrent(fileDownloadHandler);
}
This did not quite work either, so I've investigated further. I've noticed that unlike I expected, the "scheduled" request handlers would not get executed on a separate request, as I expected, but on the same one. I figured that it must be that the page gets locked for the first handler and then remains locked while the second one is executing as well. So I've attempted to force the download handler into a separate request (via an ajax behaviour):
public void startDownload(AjaxRequestTarget target) throws DownloadTargetNotFoundException{
target.appendJavaScript("setTimeout(\"window.location.href='" + getCallbackUrl() + "'\", 100);");
}
#Override
public void onRequest() {
sendFile(getFile());
logger.debug("Download initiated");
}
I've found this here and hoped it could potentially be what I've been looking for. However, unsurprisingly so, the page gets locked still (I would imagine because the behaviour still has to be retrieved from the page, for which the page lock has to be acquired).
I'm at a loss where I should be looking next, especially after all this time trying to get a simple download link working. I was considering creating another web filter one layer above wicket, which could be signaled from within wicket to create the download after the wicket filter is finished with its work (and hence the page lock is already released), but that seems a bit excessive for a task like this.
Any suggestions are welcome.

You have to download from a resource, see
http://wicketinaction.com/2012/11/uploading-files-to-wicket-iresource/ and read http://wicket.apache.org/guide/guide/resources.html

Java data object for bidirectional I/O

I am developing an interface that takes as input an encrypted byte stream -- probably a very large one -- that generates output of more or less the same format.
The input format is this:
{N byte envelope}
- encryption key IDs &c.
{X byte encrypted body}
The output format is the same.
Here's the usual use case (heavily pseudocoded, of course):
Message incomingMessage = new Message (inputStream);
ProcessingResults results = process (incomingMessage);
MessageEnvelope messageEnvelope = new MessageEnvelope ();
// set message encryption options &c. ...
Message outgoingMessage = new Message ();
outgoingMessage.setEnvelope (messageEnvelope);
writeProcessingResults (results, message);
message.writeToOutput (outputStream);
To me, it seems to make sense to use the same object to encapsulate this behaviour, but I'm at a bit of a loss as to how I should go about this. It isn't practical to load all of the encrypted body in at a time; I need to be able to stream it (so, I'll be using some kind of input stream filter to decrypt it) but at the same time I need to be able to write out new instances of this object. What's a good approach to making this work? What should Message look like internally?

I won't create one class to handle in- and output - one class, one responsibility. I would like two filter streams, one for input/decryption and one for output/encryption:
InputStream decrypted = new DecryptingStream(inputStream, decryptionParameters);
...
OutputStream encrypted = new EncryptingStream(outputSream, encryptionOptions);
They may have something like a lazy init mechanism reading the envelope before first read() call / writing the envelope before first write() call. You also use classes like Message or MessageEnvelope in the filter implementations, but they may stay package protected non API classes.
The processing will know nothing about de-/encryption just working on a stream. You may also use both streams for input and output at the same time during processing streaming the processing input and output.

Can you split the body at arbitrary locations?
If so, I would have two threads, input thread and output thread and have a concurrent queue of strings that the output thread monitors. Something like:
ConcurrentLinkedQueue<String> outputQueue = new ConcurrentLinkedQueue<String>();
...
private void readInput(Stream stream) {
String str;
while ((str = stream.readLine()) != null) {
outputQueue.put(processStream(str));
}
}
private String processStream(String input) {
// do something
return output;
}
private void writeOutput(Stream out) {
while (true) {
while (outputQueue.peek() == null) {
sleep(100);
}
String msg = outputQueue.poll();
out.write(msg);
}
}
Note: This will definitely not work as-is. Just a suggestion of a design. Someone is welcome to edit this.

If you need to read and write same time you either have to use threads (different threads reading and writing) or asynchronous I/O (the java.nio package). Using input and output streams from different threads is not a problem.
If you want to make a streaming API in java, you should usually provide InputStream for reading and OutputStream for writing. This way those can then be passed for other APIs so that you can chain things and so get the streams go all the way as streams.
Input example:
Message message = new Message(inputStream);
results = process(message.getInputStream());
Output example:
Message message = new Message(outputStream);
writeContent(message.getOutputStream());
The message needs to wrap the given streams with a classes that do the needed encryption and decryption.
Note that reading multiple messages at same time or writing multiple messages at same time would need support from the protocol too. You need to get the synchronization correct.

You should check Wikipedia article on different block cipher modes supporting encryption of streams. The different encryption algorithms may support a subset of these.
Buffered streams will allow you to read, encrypt/decrypt and write in a loop.
Examples demonstrating ZipInputStream and ZipOutputStream could provide some guidance on how you may solve this. See example.

What you need is using Cipher Streams (CipherInputStream). Here is an example of how to use it.

I agree with Arne, the data processor shouldn't know about encryption, it just needs to read the decrypted body of the message, and write out the results, and stream filters should take care of encryption. However, since this is logically operating on the same piece of information (a Message), I think they should be packaged inside one class which handles the message format, although the encryption/decryption streams are indeed independent from this.
Here's my idea for the structure, flipping the architecture around somewhat, and moving the Message class outside the encryption streams:
class Message {
InputStream input;
Envelope envelope;
public Message(InputStream input) {
assert input != null;
this.input = input;
}
public Message(Envelope envelope) {
assert envelope != null;
this.envelope = envelope;
}
public Envelope getEnvelope() {
if (envelope == null && input != null) {
// Read envelope from beginning of stream
envelope = new Envelope(input);
}
return envelope
}
public InputStream read() {
assert input != null
// Initialise the decryption stream
return new DecryptingStream(input, getEnvelope().getEncryptionParameters());
}
public OutputStream write(OutputStream output) {
// Write envelope header to output stream
getEnvelope().write(output);
// Initialise the encryption
return new EncryptingStream(output, getEnvelope().getEncryptionParameters());
}
}
Now you can use it by creating a new message for the input, and one for the output:
OutputStream output; // This is the stream for sending the message
Message inputMessage = new Message(input);
Message outputMessage = new Message(inputMessage.getEnvelope());
process(inputMessage.read(), outputMessage.write(output));
Now the process method just needs to read chunks of data as required from the input, and write results to the output:
public void process(InputStream input, OutputStream output) {
byte[] buffer = new byte[1024];
int read;
while ((read = input.read(buffer) > 0) {
// Process buffer, writing to output as you go.
}
}
This all now works in lockstep, and you don't need any extra threads. You can also abort early without having to process the whole message (if the output stream is closed for example).