Develop Reference

Read the binary image data from a URL into a ByteArrayInputStream from HttpUrlConnect::URL

I'm trying to pull the image from a URL and read it directly into a ByteArrayInputStream. I found one way of doing it, but it requires an image type, and there will be various image types, so I'd like to find a simple way to just read the binary data right in.
Here is my latest attempt. I'm using a BufferedImage, which I don't think is necessary.
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
//Read in the image
BufferedImage image = ImageIO.read(url);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(image, "png", baos);
is = new ByteArrayInputStream(baos.toByteArray());

URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
url.openStream().transferTo(baos);
ByteArrayInputStream in = new ByteArrayInputStream(baos.toByteArray());
The transferTo() method exists since Java 9. If you should use an older version of Java please see here for an alternative. Main drawback of this solution is that it has to read the whole file into memory first. If you anyway plan to forward the binary data to an other process you could omit the ByteArray streams and transfer the content directly to an OutputStream.

As an alternative to the solution proposed by #rmunge, the Apache Commons IO library provides the class IOUtils which can be vey useful in your use case.
If you are using Maven for instance, you can import the library including the following dependency in your pom.xml:
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.8.0</version>
</dependency>
Then, you can use IOUtils like this:
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
try (
InputStream imageInputStream = url.openStream();
ByteArrayOutputStream bOut = new ByteArrayOutputStream()
) {
// You can obtain a byte[] as well if required
// Please, consider write to the actual final OutputStream instead
// of into the intermediate byte array output stream to optimize memory
// consumption
IOUtils.copy(imageInputStream, bOut);
// Create an input stream from the read bytes
ByteArrayInputStream in = new ByteArrayInputStream(bOut.toByteArray());
// ...
} catch (IOException ioe) {
ioe.printStackTrace();
}
Or simply this approach:
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
byte[] imageBytes = IOUtils.toByteArray(url);
ByteArrayInputStream in = new ByteArrayInputStream(imageBytes);
For your comments, if the problem if you are trying to avoid network latency problems, if the requirement for a ByteArrayInputStream is not strictly necessary, as you can see in the javadocs perhaps the following code may be helpful as well:
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
try (InputStream imageInputStream = url.openStream()) {
InputStream in = IOUtils.toBufferedInputStream(imageInputStream);
//...
}
Of course, you can always perform the read and write "manually" using the standard Java InputStream and OutputStream mechanisms:
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
try (
InputStream inputStream = url.openStream();
BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream);
) {
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = bufferedInputStream.read(buffer)) != -1) {
bufferedOutputStream.write(buffer, 0, bytesRead);
}
bufferedOutputStream.flush();
// Create an input stream from the read bytes
ByteArrayInputStream in = new ByteArrayInputStream(outputStream.toByteArray());
// ...
} catch (IOException ioe) {
ioe.printStackTrace();
}
If you require more control about the underlying URL connection you can use URLConnection or HttpURLConnection, or many HTTP client libraries like Apache HttpClient or OkHttp, to name some of them.
Take as example the problem pointed out by #LuisCarlos in his comment, in order to avoid possible leak connections:
URLConnection urlConn = null;
try {
urlConn = url.openConnection();
urlConn.setConnectTimeout(5000);
urlConn.setReadTimeout(30000);
InputStream inputStream = urlConn.getInputStream();
// the rest of the code...
} catch (Exception e) {
}
If you need to detect the actual image type consider the use of Tika or JMimeMagic.

Here's the solution I found to work. Thanks for the two approaches above. I'd rather avoid external libraries, but because the environment is a real pain. Similar, I should have access to Java 9 and transferTo(), but that's not working.
This answerer was also helpful: Convert InputStream(Image) to ByteArrayInputStream
URL url = new URL("http://hobbylesson.com/wp-content/uploads/2015/04/Simple-Acrylic-Painting-Ideas00005.jpg");
InputStream source = url.openStream();
byte[] buf = new byte[8192];
int bytesRead = 0;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while((bytesRead = source.read(buf)) != -1) {
baos.write(buf, 0, bytesRead);
}
is = new ByteArrayInputStream(baos.toByteArray());

Blank pages in pdf after downloading it from web

I am trying to download a PDF file with HttpClient, it is downloading the PDF file but pages are blank. I can see the bytes on console from response if I print them. But when I try to write it to file it is producing a blank file.
FileUtils.writeByteArrayToFile(new File(outputFilePath), bytes);
However the file is showing correct size of 103KB and 297KB as expected but its just blank!!
I tried with Output stream as well like:
FileOutputStream fileOutputStream = new FileOutputStream(outFile);
fileOutputStream.write(bytes);
Also tried to write with UTF-8 coding like:
Writer out = new BufferedWriter( new OutputStreamWriter(
new FileOutputStream(outFile), "UTF-8"));
String str = new String(bytes, StandardCharsets.UTF_8);
try {
out.write(str);
} finally {
out.close();
}
Nothing is working for me. Any suggestion is highly appreciated..
Update: I am using DefaultHttpClient.
HttpGet httpget = new HttpGet(targetURI);
HttpResponse response = null;
String htmlContents = null;
try {
httpget = new HttpGet(url);
response = httpclient.execute(httpget);
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
...

You do
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
As has already been mentioned in comments, using a Reader class can damage binary data, e.g. PDF files. Thus, you should not wrap your content in an InputStreamReader.
As your content can be used to construct an InputStreamReader, though, I assume response.getEntity().getContent() returns an InputStream. Such an InputStream usually can be directly used as IOUtils.toByteArray argument.
So:
InputStream dataStream=response.getEntity().getContent();
byte[] bytes = IOUtils.toByteArray(dataStream);
should already work for you!

Here is a method I use to download a PDF file from a specific URL. The method requires two string arguments, an url string (example: "https://www.ibm.com/support/knowledgecenter/SSWRCJ_4.1.0/com.ibm.safos.doc_4.1/Planning_and_Installation.pdf") and a destination folder path to download the PDF file (or whatever) into. If the destination path does not exist within the local file system then it is automatically created:
public boolean downloadFile(String urlString, String destinationFolderPath) {
boolean result = false; // will turn to true if download is successful
if (!destinationFolderPath.endsWith("/") && !destinationFolderPath.endsWith("\\")) {
destinationFolderPath+= "/";
}
// If the destination path does not exist then create it.
File foldersToMake = new File(destinationFolderPath);
if (!foldersToMake.exists()) {
foldersToMake.mkdirs();
}
try {
// Open Connection
URL url = new URL(urlString);
// Get just the file Name from URL
String fileName = new File(url.getPath()).getName();
// Try with Resources....
try (InputStream in = url.openStream(); FileOutputStream outStream =
new FileOutputStream(new File(destinationFolderPath + fileName))) {
// Read from resource and write to file...
int length = -1;
byte[] buffer = new byte[1024]; // buffer for portion of data from connection
while ((length = in.read(buffer)) > -1) {
outStream.write(buffer, 0, length);
}
}
// File Successfully Downloaded");
result = true;
}
catch (MalformedURLException ex) { ex.printStackTrace(); }
catch (IOException ex) { ex.printStackTrace(); }
return result;
}

Convert byte array to playable mp4 file

I need to do some processing on the raw file bytes and the algorithm is fine with working on .wav file but the problem is the .mp4 files it is not playable at all.
I think the problem that the file contains raw bytes only and no header
please help me
public void convert(String maskFlag, File endFile)
{
byte[] rawData = getarr(originalFile);
byte[] effectedData = rawData.clone();
effectedData = changePitch(rawData, parameters.getPitch());
effectedData = changeVolume(effectedData, parameters.getVolume());
effectedData = changeSpeed(effectedData, parameters.getSpeed());
FileOutputStream out = new FileOutputStream(endFile);
out.write(effectedData);
out.close();
}
byte[] getarr(File file) throws Exception
{
InputStream fis = new FileInputStream(file);
fis = new FileInputStream(file);
byte [] byteArr = IOUtils.toByteArray(fis);
return byteArr;
}

Insufficient System resources exist to complete the requested services

Getting the above error when trying to download large data using HttpGet
String uri = "";
getMethod = executeGet(uri);
httpClient.executeMethod(getMethod);
InputStream istream = getMethod.getResponseBodyAsStream();
byte[] data = IOUtils.toByteArray(istream);
FileUtils.writeByteArraytoFile(new File("xxx.zip"),data)

You are using a temporary byte array that might be the cause of the problem.
You can directly write the content of the stream to your file.
String uri = "";
getMethod = executeGet(uri);
httpClient.executeMethod(getMethod);
InputStream istream = getMethod.getResponseBodyAsStream();
IOUtils.copy(istream, new FileOutputStream(new File("xxx.zip"));

You're reading the entire response into the byte[] (memory). Instead, you could stream the output as you read it from istream with something like,
File f = new File("xxx.zip");
try (OutputStream os = new BufferedOutputStream(new FileOutputStream(f));) {
int c = -1;
while ((c = istream.read()) != -1) {
os.write(c);
}
} catch (Exception e) {
e.printStackTrace();
}

How to get a binary file from a remote php script response?

I'm calling a script that gives me a binary file (12345.cl), with binary data. The script is done, and it's working, if I paste it on the navigator I get the binary file.
Now I have a problem: How I transform the response of the script into a binary resource to use it in my app?
For the moment, i have this code:
public void decodeStream( String mURL ){
BufferedInputStream bis = new BufferedInputStream(new URL(mURL).openStream(), BUFFER_IO_SIZE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(baos, BUFFER_IO_SIZE);
copy(bis, bos);
bos.flush();
Then, I have a BufferedOutputStream with the response, but I don't know how to transform it into a binary resource to use it
I need to obtain a datainputstream with the file but I don't know how to achieve it

You can use following code:
public void decodeStream( String mURL, String ofile ) throws Exception {
InputStream in = null;
FileOutputStream out = null;
try {
URL url = new URL(mURL);
URLConnection urlConn = url.openConnection();
in = urlConn.getInputStream();
out = new FileOutputStream(ofile);
int c;
byte[] b = new byte[1024];
while ((c = in.read(b)) != -1)
out.write(b, 0, c);
} finally {
if (in != null)
in.close();
if (out != null)
out.close();
}
}