Develop Reference

Sharing shared Memory File descriptors across android app/processes using binder

We have two processes which interacts through shared memory to read/write data. One process is application(client) and other process is background service which is separate application in this example it is ashmem producer service.
AshmemService :- This is background running service which is sticky, this provides API to create shared memory channel using android MemoryFile API. This service creates shared memory and get file descriptor and that fd is passed to client process through binder interface.
AshmemClient :- This is application which get shared memory channel file descriptor from AshmemService. This client app binds to service, once it is connected it calls aidl interface api to get fd.
Channel creation works fine which creates shared memory , Now for testing purpose I am writing block of bytes from Client to shared memory. once writing is done then I am trying to same block in service to valid data received from app.
From Client app whenever I am trying to write block of 128 bytes, writes returns 0 so looks like I am not able to write in FD which is received through parcelable binder. Even I have verified that fd is valid through fd.valid() API.
pfd.getStatSize() returns -1. I think I am not getting correct fd from parcelable file descriptor.
http://developer.android.com/reference/android/os/Parcel.html says that
The returned file descriptor is a dup of the original file descriptor: the object and fd is different, but operating on the same underlying file stream, with the same position, etc.
In Reverse case, which is in service where we have created shared memory if we try to write byte block then same block we can read in application that I have validated and it is working as expected.
package com.example.service.ashmem;
import java.io.FileDescriptor;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.FileOutputStream;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;
import android.os.Parcel;
import android.os.ParcelFileDescriptor;
import android.os.Parcelable;
import android.util.Log;
public class ConnectionWriter implements Parcelable {
public final static String TAG = "ConnectionWriter";
protected FileOutputStream fos = null;
protected FileDescriptor fd = null;
protected int noofBlocks = 0;
protected int blocksize = 0;
protected int lastPosition;
protected int chanId;
public ConnectionWriter(MemoryFileChannel channel, int chanId) {
this.fd = channel.getFileDescriptor();
channel.getParcelFileDescriptor();
this.chanId = chanId;
this.blocksize = channel.getBlocksize();
this.noofBlocks = channel.getNoofBlocks();
Log.d(TAG, "ctor: chanId: " + chanId + " fd valid: " + fd.valid()+ " fd: "+fd);
}
public ConnectionWriter(Parcel in) {
readFromParcel(in);
}
public static final Parcelable.Creator<ConnectionWriter> CREATOR = new
Parcelable.Creator<ConnectionWriter>() {
public ConnectionWriter createFromParcel(Parcel in) {
Log.d(TAG, "new ConnectionWriter");
return new ConnectionWriter(in);
}
#Override
public ConnectionWriter[] newArray(int arg0) {
return null;
}
};
public boolean writeBlock(ByteBuffer data, int pos)
throws IllegalArgumentException, IOException {
FileChannel fc = fos.getChannel();
Log.d(TAG, "ByteBuffer capacity : "+data.capacity());
Log.d(TAG, "location to write : "+pos*blocksize);
Log.d(TAG, "FileChannel : "+fc.isOpen() + fc.position() + fc.size());
int len = fc.write(data);
Log.d(TAG, "block write in ashmem : "+len + " fd: "+fd);
return true;
}
public void writeToParcel(Parcel out, int arg1) {
Log.d(TAG, "writeToParcel "+ "fd: "+fd + " fd valid"+ fd.valid());
out.writeInt(blocksize);
out.writeInt(noofBlocks);
out.writeFileDescriptor(fd);
out.writeInt(lastPosition);
out.writeInt(chanId);
}
public void readFromParcel(Parcel in) {
// Make sure to validate data since these are coming from an external process
blocksize = in.readInt();
noofBlocks = in.readInt();
ParcelFileDescriptor pfd = in.readFileDescriptor();
if (pfd != null) {
fd = pfd.getFileDescriptor();
Log.d(TAG, "readFromParcel "+ "fd: "+fd);
fos = new FileOutputStream(fd);
Log.v(TAG, " fd is " + fd.valid());
long stat = pfd.getStatSize();
Log.d(TAG, " fd stat " + stat);
}
lastPosition = in.readInt();
chanId = in.readInt();
}
public int getId() {
return chanId;
}
#Override
public int describeContents() {
// TODO Auto-generated method stub
return 0;
}
public void close() {
try {
if (fos != null)
fos.close();
} catch (IOException ioe) {
}
}
}
Below are logs on adb logcat
ConnectionWriter(31694): new ConnectionWriter
ConnectionWriter(31694): readFromParcel fd: FileDescriptor[58]
ConnectionWriter(31694): fd is true
ConnectionWriter(31694): fd stat -1
ConnectionWriter(31694): ByteBuffer capacity : 128
ConnectionWriter(31694): location to write : 0
ConnectionWriter(31694): FileChannel : true00
ConnectionWriter(31694): block write in ashmem : 0 fd: FileDescriptor[58]
I am not sure but looks like I am not getting correct ParcelFileDescriptor. If anybody has any suggestion about sharing MemoryFile fd. please suggest.

Send object over udp in Java

I'm trying to send and object over udp by first serializing it and then deserializing it on the other end. I thought this would be trivial since I have sent other data over udp before and serialized stuff to the files etc.
I have debugged thing some time now and I keep getting EOFException on the receiving end. Packets arrive properly but somehow deserialization fails. I'm not sure if the mistake is in sender or receiver. I suppose the problem might be about the receiver not knowing the size of the packet.
Here is my sender class:
package com.machinedata.sensordata;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import android.content.Context;
import android.util.Log;
import com.machinedata.io.DataSerializer;
import com.machinedata.io.ManagerUdpPacket;
/**
* This class sends udp-packets. It is used to send driver's information to the manager tablet.
* #author tuomas
*
*/
public class UdpSender
{
private final int MANAGER_PORT = 1234;
private String ip = "192.168.11.50"; //tablet's IP
private DatagramSocket sock = null;
private InetAddress host;
private String mType;
private DataSerializer dataser;
public UdpSender(Context context)
{
try
{
sock = new DatagramSocket();
host = InetAddress.getByName(ip); //tabletin ip
}
catch(Exception e)
{
System.err.println("Exception alustettaessa senderia" + e);
}
dataser = new DataSerializer(context);
}
/**
* With this function we can send packets about our machine to the manager to
* see in the fleet-view.
*/
public void sendToManager(ManagerUdpPacket managerUdp)
{
//serialize
Log.v("sendudp", "Send a packet: " + managerUdp.getDriver());
//serialize
byte[] data = dataser.serializeManagerPacket(managerUdp);
//send
try
{
DatagramPacket dp = new DatagramPacket(data , data.length , host , MANAGER_PORT);
sock.send(dp);
}
catch(IOException e)
{
System.err.println("IOException senderissa " + e);
}
}
public void close()
{
sock.close();
}
}
Here is the serialization function:
/**
* Serializes packet to be sent over udp to the manager tablet.
*/
public byte[] serializeManagerPacket(ManagerUdpPacket mp)
{
try
{
ByteArrayOutputStream baos = new ByteArrayOutputStream(2048);
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(mp);
oos.close();
// get the byte array of the object
byte[] obj= baos.toByteArray();
baos.close();
return obj;
}
catch(Exception e) {
e.printStackTrace();
}
return null;
}
Packet receiver class
public class UdpReceiver {
private DatagramSocket clientSocket;
private byte[] receiveData;
private final int timeout = 1;
/**
* Create a receiver.
* #param port Port to receive from.
* #param signCount Number of signals in a packet
*/
public UdpReceiver(int port)
{
//receiveData = serializeManagerPacket(new ManagerUdpPacket("asd", new MachineData(1, 2, "asd", "modelName"), 1,2,3,4,5.0,null));
try{
clientSocket=new DatagramSocket(port);
clientSocket.setReceiveBufferSize(2048);
clientSocket.setSoTimeout(timeout);
}catch(SocketException e){
Log.e("ERR", "SocketException in UdpReceiver()");
}
}
public void close()
{
clientSocket.close();
}
/**
* Receive a data packet and split it into array.
* #param data Array to put data in, must be correct size
* #return True on successful read, false otherwise
*/
public ManagerUdpPacket receive()
{
//receive a packet
DatagramPacket recvPacket = new DatagramPacket(receiveData, receiveData.length);
try{
clientSocket.receive(recvPacket);
}catch(IOException e){
Log.e("ERR", "IOException in UdpReceiver.receive");
return null;
}
ManagerUdpPacket obj = deserializeManagerPacket(receiveData);
if (obj != null)
Log.v("udpPacket", "UDP saatu: " + obj.getDriver());
return obj;
}
/**
* Deserialize the udp-packet back to readable data.
* #param data
* #return
*/
public ManagerUdpPacket deserializeManagerPacket(byte[] data)
{
try
{
ObjectInputStream iStream = new ObjectInputStream(new ByteArrayInputStream(data));
ManagerUdpPacket obj = (ManagerUdpPacket) iStream.readObject();
iStream.close();
return obj;
}
catch(Exception e)
{
e.printStackTrace();
}
return null;
}
}
Thread which listens packets in receiving end:
dataStreamTask = new TimerTask()
{
public void run()
{
if (currentlyStreaming)
{
ManagerUdpPacket mp = udpReceiver.receive();
if(mp != null)
{
Log.v("log", "Paketti saatu! " + mp.getDriver());
}
//stop thread until next query
try {
synchronized(this){
this.wait(queryInterval);
}
} catch (InterruptedException e) {
Log.e("ERR", "InterruptedException in TimerTask.run");
}
}
}
And finally the class I'm sending over the UDP:
public class ManagerUdpPacket implements Serializable
{
private static final long serialVersionUID = 9169314425496496555L;
private Location gpsLocation;
private double totalFuelConsumption;
private long operationTime;
//workload distribution
private long idleTime = 0;
private long normalTime = 0;
private long fullTime = 0;
private int currentTaskId;
private String driverName;
String machineModelName = "";
String machineName = "";
int machineIconId = -1;
int machinePort = -1;
public ManagerUdpPacket(String driver, MachineData machine, int currentTaskId, long idleTime, long fullTime, long operationTime, double fuelConsumption, Location location)
{
driverName = driver;
this.currentTaskId = currentTaskId;
this.idleTime = idleTime;
this.fullTime = fullTime;
this.operationTime = operationTime;
this.totalFuelConsumption = fuelConsumption;
this.gpsLocation = location;
machineModelName = machine.getModelName();
machineName = machine.getName();
machineIconId = machine.getIconId();
machinePort = machine.getPort();
}
public String getDriver()
{
return driverName;
}
public int getCurrentTaskId()
{
return currentTaskId;
}
public long getIdleTime()
{
return idleTime;
}
public long getFullTime()
{
return fullTime;
}
public long getOperationTime()
{
return operationTime;
}
public double getTotalFuelConsumption()
{
return totalFuelConsumption;
}
public double getLocation()
{
return gpsLocation.getLatitude();
}
public String getMachineModelName()
{
return machineModelName;
}
public String getMachineName()
{
return machineName;
}
public int getMachineIconId()
{
return machineIconId;
}
public int getMachinePort()
{
return machinePort;
}
}
I tried to get the packet size from the size of the serialized packet or inserting arbitrary 2048 based on some examples on internet. Couldn't get it work though.

As far as i know the receive function returns the length of the bytes it received. But your buffer will be full:
Example:
int buffersize = 1024;
You send 8bytes over udp.
So your byte[] will be full with your 8 bytes but the rest of the 1024 will be 0.
save the size you get by the .receive() call and just save all values of your buffer to another byte[] and you should get your object.
For your example:
public ManagerUdpPacket receive()
{
int receivedBytes = 0;
//receive a packet
DatagramPacket recvPacket = new DatagramPacket(receiveData, receiveData.length);
try{
receivedBytes = clientSocket.receive(recvPacket);
}catch(IOException e){
Log.e("ERR", "IOException in UdpReceiver.receive");
return null;
}
byte[] myObject = new byte[receivedBytes];
for(int i = 0; i < receivedBytes; i++)
{
myObject[i] = receiveData[i];
}
ManagerUdpPacket obj = deserializeManagerPacket(myObject);
if (obj != null)
Log.v("udpPacket", "UDP saatu: " + obj.getDriver());
return obj;
}

When receiving data on UDP, always use java.net.DatagramSocket.getReceiveBufferSize();. This is the actual size of the platform or SP_RCVBUF for the socket. Since UDP is a datagram based protocol unlike TCP, which is streaming protocol, receiving buffers become critical for data sanity. Usually, receiving and sending buffers are equal in size, but you are not bothered while sending when using DatagramSocket.send(DatagramPacket), alternately, you can also use DatagramSocket.setSendBufferSize(DatagramSocket.getSendBufferSize()) for using the SO_SNDBUF option for this socket. Keep in mind, in UDP, if you use a SO_SNDBUF size greater than platform's, the packet can be discarded.

How to serve static content using suns simple httpserver

I'm using jersey's HttpServerFactory to create a simple embedded HttpServer that hosts a couple of rest services. We just needed something small quick and lightweight. I need to host a small static html page inside the same server instance. Is there a simple way to add a static handler to the server? Is there a pre-defined handler I can use? It seems like a pretty common task, I'd hate to re-write code for it if it already exists.
server = HttpServerFactory.create(url);
server.setExecutor(Executors.newCachedThreadPool());
server.createContext("/staticcontent", new HttpHandler() {
#Override
public void handle(HttpExchange arg0) throws IOException {
//What goes here?
}
});
server.start();

Here is a safe version. You may want to add a couple of MIME types, depending on which ones are common (or use another method if your platform has that).
package de.phihag.miniticker;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.Map;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class StaticFileHandler implements HttpHandler {
private static final Map<String,String> MIME_MAP = new HashMap<>();
static {
MIME_MAP.put("appcache", "text/cache-manifest");
MIME_MAP.put("css", "text/css");
MIME_MAP.put("gif", "image/gif");
MIME_MAP.put("html", "text/html");
MIME_MAP.put("js", "application/javascript");
MIME_MAP.put("json", "application/json");
MIME_MAP.put("jpg", "image/jpeg");
MIME_MAP.put("jpeg", "image/jpeg");
MIME_MAP.put("mp4", "video/mp4");
MIME_MAP.put("pdf", "application/pdf");
MIME_MAP.put("png", "image/png");
MIME_MAP.put("svg", "image/svg+xml");
MIME_MAP.put("xlsm", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
MIME_MAP.put("xml", "application/xml");
MIME_MAP.put("zip", "application/zip");
MIME_MAP.put("md", "text/plain");
MIME_MAP.put("txt", "text/plain");
MIME_MAP.put("php", "text/plain");
};
private String filesystemRoot;
private String urlPrefix;
private String directoryIndex;
/**
* #param urlPrefix The prefix of all URLs.
* This is the first argument to createContext. Must start and end in a slash.
* #param filesystemRoot The root directory in the filesystem.
* Only files under this directory will be served to the client.
* For instance "./staticfiles".
* #param directoryIndex File to show when a directory is requested, e.g. "index.html".
*/
public StaticFileHandler(String urlPrefix, String filesystemRoot, String directoryIndex) {
if (!urlPrefix.startsWith("/")) {
throw new RuntimeException("pathPrefix does not start with a slash");
}
if (!urlPrefix.endsWith("/")) {
throw new RuntimeException("pathPrefix does not end with a slash");
}
this.urlPrefix = urlPrefix;
assert filesystemRoot.endsWith("/");
try {
this.filesystemRoot = new File(filesystemRoot).getCanonicalPath();
} catch (IOException e) {
throw new RuntimeException(e);
}
this.directoryIndex = directoryIndex;
}
/**
* Create and register a new static file handler.
* #param hs The HTTP server where the file handler will be registered.
* #param path The path in the URL prefixed to all requests, such as "/static/"
* #param filesystemRoot The filesystem location.
* For instance "/var/www/mystaticfiles/".
* A request to "/static/x/y.html" will be served from the filesystem file "/var/www/mystaticfiles/x/y.html"
* #param directoryIndex File to show when a directory is requested, e.g. "index.html".
*/
public static void create(HttpServer hs, String path, String filesystemRoot, String directoryIndex) {
StaticFileHandler sfh = new StaticFileHandler(path, filesystemRoot, directoryIndex);
hs.createContext(path, sfh);
}
public void handle(HttpExchange he) throws IOException {
String method = he.getRequestMethod();
if (! ("HEAD".equals(method) || "GET".equals(method))) {
sendError(he, 501, "Unsupported HTTP method");
return;
}
String wholeUrlPath = he.getRequestURI().getPath();
if (wholeUrlPath.endsWith("/")) {
wholeUrlPath += directoryIndex;
}
if (! wholeUrlPath.startsWith(urlPrefix)) {
throw new RuntimeException("Path is not in prefix - incorrect routing?");
}
String urlPath = wholeUrlPath.substring(urlPrefix.length());
File f = new File(filesystemRoot, urlPath);
File canonicalFile;
try {
canonicalFile = f.getCanonicalFile();
} catch (IOException e) {
// This may be more benign (i.e. not an attack, just a 403),
// but we don't want the attacker to be able to discern the difference.
reportPathTraversal(he);
return;
}
String canonicalPath = canonicalFile.getPath();
if (! canonicalPath.startsWith(filesystemRoot)) {
reportPathTraversal(he);
return;
}
FileInputStream fis;
try {
fis = new FileInputStream(canonicalFile);
} catch (FileNotFoundException e) {
// The file may also be forbidden to us instead of missing, but we're leaking less information this way
sendError(he, 404, "File not found");
return;
}
String mimeType = lookupMime(urlPath);
he.getResponseHeaders().set("Content-Type", mimeType);
if ("GET".equals(method)) {
he.sendResponseHeaders(200, canonicalFile.length());
OutputStream os = he.getResponseBody();
copyStream(fis, os);
os.close();
} else {
assert("HEAD".equals(method));
he.sendResponseHeaders(200, -1);
}
fis.close();
}
private void copyStream(InputStream is, OutputStream os) throws IOException {
byte[] buf = new byte[4096];
int n;
while ((n = is.read(buf)) >= 0) {
os.write(buf, 0, n);
}
}
private void sendError(HttpExchange he, int rCode, String description) throws IOException {
String message = "HTTP error " + rCode + ": " + description;
byte[] messageBytes = message.getBytes("UTF-8");
he.getResponseHeaders().set("Content-Type", "text/plain; charset=utf-8");
he.sendResponseHeaders(rCode, messageBytes.length);
OutputStream os = he.getResponseBody();
os.write(messageBytes);
os.close();
}
// This is one function to avoid giving away where we failed
private void reportPathTraversal(HttpExchange he) throws IOException {
sendError(he, 400, "Path traversal attempt detected");
}
private static String getExt(String path) {
int slashIndex = path.lastIndexOf('/');
String basename = (slashIndex < 0) ? path : path.substring(slashIndex + 1);
int dotIndex = basename.lastIndexOf('.');
if (dotIndex >= 0) {
return basename.substring(dotIndex + 1);
} else {
return "";
}
}
private static String lookupMime(String path) {
String ext = getExt(path).toLowerCase();
return MIME_MAP.getOrDefault(ext, "application/octet-stream");
}
}

This will do the trick, though it does allow anyone to walk the tree by requesting ../../../
You can change ./wwwroot to any valid java filepath.
static class MyHandler implements HttpHandler {
public void handle(HttpExchange t) throws IOException {
String root = "./wwwroot";
URI uri = t.getRequestURI();
System.out.println("looking for: "+ root + uri.getPath());
String path = uri.getPath();
File file = new File(root + path).getCanonicalFile();
if (!file.isFile()) {
// Object does not exist or is not a file: reject with 404 error.
String response = "404 (Not Found)\n";
t.sendResponseHeaders(404, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
} else {
// Object exists and is a file: accept with response code 200.
String mime = "text/html";
if(path.substring(path.length()-3).equals(".js")) mime = "application/javascript";
if(path.substring(path.length()-3).equals("css")) mime = "text/css";
Headers h = t.getResponseHeaders();
h.set("Content-Type", mime);
t.sendResponseHeaders(200, 0);
OutputStream os = t.getResponseBody();
FileInputStream fs = new FileInputStream(file);
final byte[] buffer = new byte[0x10000];
int count = 0;
while ((count = fs.read(buffer)) >= 0) {
os.write(buffer,0,count);
}
fs.close();
os.close();
}
}
}

Commons HttpClient getResponse takes very long 200 seconds

I changed nothing in this method but suddenly it takes very long. The code example below produces this error.
EDIT: i extracted the method in a single java application and tried to load the file and i have the same timeout. The 3 or 4. time he goes through this loop HttpMethodBase:691 he stops for 500 seconds at my local pc and the thread is sleeping. After sleeping the next line is outstream.close();
while ((len = instream.read(buffer)) > 0) {
outstream.write(buffer, 0, len);
}
EDIT: Here is the example code if you wanna try it at home :) (httpClient 3.1)
import java.io.IOException;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.GetMethod;
public class TestLoadImage {
/**
* #param args
*/
public static void main(String[] args) {
byte[] image = loadPhotoFromUrl("https://fbcdn-sphotos-a.akamaihd.net/photos-ak-snc1/v2635/232/115/68310606562/n68310606562_2255479_948765.jpg");
System.out.println(image.length);
}
private static class GetMethodIgnoringContentLength extends GetMethod {
public GetMethodIgnoringContentLength(String uri) {
super(uri);
}
#Override
public long getResponseContentLength() {
// ignores content-length header, fakes "not specified":
return -1;
}
}
public static byte[] loadPhotoFromUrl(String photoUrl) {
HttpClient httpClient = new HttpClient();
httpClient.getParams().setBooleanParameter("http.connection.stalecheck", true);
GetMethod get = new GetMethodIgnoringContentLength(photoUrl);
try {
int httpStatus = httpClient.executeMethod(get);
if (httpStatus == HttpStatus.SC_OK) {
byte[] imageBytes = get.getResponseBody();
if (imageBytes.length > 0) {
return imageBytes;
} else {
System.out.println("Failed: empty response/zero data");
}
} else {
System.out.println("Failed: "+ httpStatus);
}
} catch (IOException ioe) {
System.out.println( ioe);
} finally {
get.releaseConnection();
}
return null;
}
}

Fixed it with an update to HttpClient4.1 (and some refactoring required therefor). But if someone knows the reason in the example above i switch the correct answer.

Download file using java apache commons?

How can I use the library to download a file and print out bytes saved? I tried using
import static org.apache.commons.io.FileUtils.copyURLToFile;
public static void Download() {
URL dl = null;
File fl = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
copyURLToFile(dl, fl);
} catch (Exception e) {
System.out.println(e);
}
}
but I cannot display bytes or a progress bar. Which method should I use?
public class download {
public static void Download() {
URL dl = null;
File fl = null;
String x = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
CountingOutputStream count = new CountingOutputStream(os);
dl.openConnection().getHeaderField("Content-Length");
IOUtils.copy(is, os);//begin transfer
os.close();//close streams
is.close();//^
} catch (Exception e) {
System.out.println(e);
}
}

If you are looking for a way to get the total number of bytes before downloading, you can obtain this value from the Content-Length header in http response.
If you just want the final number of bytes after the download, it is easiest to check the file size you just write to.
However if you want to display the current progress of how many bytes have been downloaded, you might want to extend apache CountingOutputStream to wrap the FileOutputStream so that everytime the write methods are called it counts the number of bytes passing through and update the progress bar.
Update
Here is a simple implementation of DownloadCountingOutputStream. I am not sure if you are familiar with using ActionListener or not but it is a useful class for implementing GUI.
public class DownloadCountingOutputStream extends CountingOutputStream {
private ActionListener listener = null;
public DownloadCountingOutputStream(OutputStream out) {
super(out);
}
public void setListener(ActionListener listener) {
this.listener = listener;
}
#Override
protected void afterWrite(int n) throws IOException {
super.afterWrite(n);
if (listener != null) {
listener.actionPerformed(new ActionEvent(this, 0, null));
}
}
}
This is the usage sample :
public class Downloader {
private static class ProgressListener implements ActionListener {
#Override
public void actionPerformed(ActionEvent e) {
// e.getSource() gives you the object of DownloadCountingOutputStream
// because you set it in the overriden method, afterWrite().
System.out.println("Downloaded bytes : " + ((DownloadCountingOutputStream) e.getSource()).getByteCount());
}
}
public static void main(String[] args) {
URL dl = null;
File fl = null;
String x = null;
OutputStream os = null;
InputStream is = null;
ProgressListener progressListener = new ProgressListener();
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
os = new FileOutputStream(fl);
is = dl.openStream();
DownloadCountingOutputStream dcount = new DownloadCountingOutputStream(os);
dcount.setListener(progressListener);
// this line give you the total length of source stream as a String.
// you may want to convert to integer and store this value to
// calculate percentage of the progression.
dl.openConnection().getHeaderField("Content-Length");
// begin transfer by writing to dcount, not os.
IOUtils.copy(is, dcount);
} catch (Exception e) {
System.out.println(e);
} finally {
IOUtils.closeQuietly(os);
IOUtils.closeQuietly(is);
}
}
}

commons-io has IOUtils.copy(inputStream, outputStream). So:
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
IOUtils.copy(is, os);
And IOUtils.toByteArray(is) can be used to get the bytes.
Getting the total number of bytes is a different story. Streams don't give you any total - they can only give you what is currently available in the stream. But since it's a stream, it can have more coming.
That's why http has its special way of specifying the total number of bytes. It is in the response header Content-Length. So you'd have to call url.openConnection() and then call getHeaderField("Content-Length") on the URLConnection object. It will return the number of bytes as string. Then use Integer.parseInt(bytesString) and you'll get your total.