Develop Reference

Blank pages in pdf after downloading it from web

I am trying to download a PDF file with HttpClient, it is downloading the PDF file but pages are blank. I can see the bytes on console from response if I print them. But when I try to write it to file it is producing a blank file.
FileUtils.writeByteArrayToFile(new File(outputFilePath), bytes);
However the file is showing correct size of 103KB and 297KB as expected but its just blank!!
I tried with Output stream as well like:
FileOutputStream fileOutputStream = new FileOutputStream(outFile);
fileOutputStream.write(bytes);
Also tried to write with UTF-8 coding like:
Writer out = new BufferedWriter( new OutputStreamWriter(
new FileOutputStream(outFile), "UTF-8"));
String str = new String(bytes, StandardCharsets.UTF_8);
try {
out.write(str);
} finally {
out.close();
}
Nothing is working for me. Any suggestion is highly appreciated..
Update: I am using DefaultHttpClient.
HttpGet httpget = new HttpGet(targetURI);
HttpResponse response = null;
String htmlContents = null;
try {
httpget = new HttpGet(url);
response = httpclient.execute(httpget);
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
...

You do
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
As has already been mentioned in comments, using a Reader class can damage binary data, e.g. PDF files. Thus, you should not wrap your content in an InputStreamReader.
As your content can be used to construct an InputStreamReader, though, I assume response.getEntity().getContent() returns an InputStream. Such an InputStream usually can be directly used as IOUtils.toByteArray argument.
So:
InputStream dataStream=response.getEntity().getContent();
byte[] bytes = IOUtils.toByteArray(dataStream);
should already work for you!

Here is a method I use to download a PDF file from a specific URL. The method requires two string arguments, an url string (example: "https://www.ibm.com/support/knowledgecenter/SSWRCJ_4.1.0/com.ibm.safos.doc_4.1/Planning_and_Installation.pdf") and a destination folder path to download the PDF file (or whatever) into. If the destination path does not exist within the local file system then it is automatically created:
public boolean downloadFile(String urlString, String destinationFolderPath) {
boolean result = false; // will turn to true if download is successful
if (!destinationFolderPath.endsWith("/") && !destinationFolderPath.endsWith("\\")) {
destinationFolderPath+= "/";
}
// If the destination path does not exist then create it.
File foldersToMake = new File(destinationFolderPath);
if (!foldersToMake.exists()) {
foldersToMake.mkdirs();
}
try {
// Open Connection
URL url = new URL(urlString);
// Get just the file Name from URL
String fileName = new File(url.getPath()).getName();
// Try with Resources....
try (InputStream in = url.openStream(); FileOutputStream outStream =
new FileOutputStream(new File(destinationFolderPath + fileName))) {
// Read from resource and write to file...
int length = -1;
byte[] buffer = new byte[1024]; // buffer for portion of data from connection
while ((length = in.read(buffer)) > -1) {
outStream.write(buffer, 0, length);
}
}
// File Successfully Downloaded");
result = true;
}
catch (MalformedURLException ex) { ex.printStackTrace(); }
catch (IOException ex) { ex.printStackTrace(); }
return result;
}

How to convert loaded bytes into image without ImageIO.read()

I can't use ImageIO.read() because of my own restrictions. I can only load bytes after GET request and I need to save this bytes to file as image. But it seems to me, that there also loads some extra data, which browser usually filter (maybe response headers). So I get the array of raw bytes which I even can't open as image.
What should I do with this bytes?
Example:
byte[] buf = ContentLoader.loadBytes(new URL("http://images.visitcanberra.com.au/images/canberra_hero_image.jpg"));
try {
FileOutputStream fileOutputStream = new FileOutputStream(new File("D:\\image.jpg"));
fileOutputStream.write(buf);
fileOutputStream.flush();
} catch (IOException e) {
e.printStackTrace();
}
loadBytes() method:
public static byte[] loadBytes(URL url) {
ByteArrayOutputStream boutArray = new ByteArrayOutputStream();
try {
URLConnection connection = url.openConnection();
BufferedInputStream bin = new BufferedInputStream(connection.getInputStream());
byte[] buffer = new byte[1024 * 16];
while (bin.read(buffer) != -1) {
boutArray.write(buffer);
boutArray.flush();
}
bin.close();
} catch (Exception e) {
return null;
}
return boutArray.toByteArray();
}

Usual problems. The standard way to copy a stream in Java is:
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
out.close();
in.close();
Note that you need to store the result returned by read() into a variable; that you need to use it in the next write() call; that you shouldn't flush() inside a loop; and that you need to close the input and output streams.
And why you're using a ByteArrayInputStream at all is a mystery. It's just a waste of time and space. Read directly from the URL input stream, and write directly to the FileOutputStream.

The following code works for me:-
URL url = new URL("my url...");
InputStream is = url.openStream();
OutputStream os = new FileOutputStream("img.jpg");
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();

Insufficient System resources exist to complete the requested services

Getting the above error when trying to download large data using HttpGet
String uri = "";
getMethod = executeGet(uri);
httpClient.executeMethod(getMethod);
InputStream istream = getMethod.getResponseBodyAsStream();
byte[] data = IOUtils.toByteArray(istream);
FileUtils.writeByteArraytoFile(new File("xxx.zip"),data)

You are using a temporary byte array that might be the cause of the problem.
You can directly write the content of the stream to your file.
String uri = "";
getMethod = executeGet(uri);
httpClient.executeMethod(getMethod);
InputStream istream = getMethod.getResponseBodyAsStream();
IOUtils.copy(istream, new FileOutputStream(new File("xxx.zip"));

You're reading the entire response into the byte[] (memory). Instead, you could stream the output as you read it from istream with something like,
File f = new File("xxx.zip");
try (OutputStream os = new BufferedOutputStream(new FileOutputStream(f));) {
int c = -1;
while ((c = istream.read()) != -1) {
os.write(c);
}
} catch (Exception e) {
e.printStackTrace();
}

OutOfMemory when creating Base64 string in java?

I used ostermillerutils library to create base64 string but I get OutOfMemory error if the image is heavy. If the image I try to convert is a simple image, the code is working fine.
public String createBase64String(InputStream in) {
//collect = new ByteArrayOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for(int readNum; (readNum = in.read(buf)) != -1; ) {
bos.write(buf, 0, readNum);
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
return null;
}
finally {
if (in != null) {
try {
in.close();
}
catch (IOException ex) {
;
}
}
}
byte[] ba = bos.toByteArray();
String coded = Base64.encodeToString(ba);
return coded;
}
I also tried doing this but the base64 was incorrect when I tried to decode it.
public void createBase64String(InputStream in) throws IOException {
//collect = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int readNum = 0;
try {
while((readNum = in.read(buf)) != -1)
{
smtp.addBase64(Base64.encodeBase64String(buf));
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
}
finally {
if (in != null) {
in.close();
}
}
}
Please suggest solutions for JDK 1.4 and also for later versions of Java.

If you like to write the encoded content straight into a file then use the following code
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
IOUtils class from Apache Commons IO.
EDIT
Since you want to do it using BufferedWriter, use it as follows
OutputStream out = Base64OutputStream(smtpSocket.getOutputStream());
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
IOUtils.copy(is, bw);

It sounds like the problem is that you're having to manipulate too much data in memory when you read the entire image. One fix would be to increase the Java heap size until you have enough memory, but that would just be avoiding the problem rather than solving it.
A better option would be to look at a streaming implementation of a Base64 encoder. This would mean you're only working on a subset of the image at any time. I believe that Base64OutputStream from Apache Commons would do the job for you.

I've fixed my problem by using javabase64-1.3.1.jar library.
OutputStream fos2 = FileUtil.getOutputStream(base64FileName, FileUtil.HDD);
InputStream in2 = FileUtil.getInputStream(fileName, FileUtil.HDD);
Base64.encode(in2, fos2);
in2.close();
fos2.close();
I stored the base64 string to a text file first.
public void createBase64String(InputStream in) throws IOException {
baos = new ByteArrayOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int readNum = 0;
smtp.addBase64("\t\t");
try {
while ((readNum = in.read(buf)) >= 0) {
baos.write(buf, 0, readNum);
smtp.addBase64(baos.toString());
baos.reset();
}
}
catch (IOException ex) {
LogUtil.error("Sending of Base64 String to SMTP: IOException: " + ex);
}
finally {
if (in != null) {
in.close();
baos.close();
}
}
baos = null;
buf = null;
}
then send each line to smtp's socket outputstream.

From Java 8 onwards, there is a simple way to implement base64 encoding in an output stream with one line of code and no external dependencies:
import java.util.Base64;
OutputStream os = ...
OutputStream base64 = Base64.getEncoder().wrap(os);
Base64 also provides other flavors of base64 encoder; see javadocs:
Base64
Base64.Encoder.wrap

How to reference a File in raw folder in Android

I just want to create a File object like this
File myImageFile = new File ("image1") ;
but it is giving me exception of FileNotFoundException
How can i reference a file inside my raw Folder
EDIT:
Actually i wanted to do something like this
MultipartEntity multipartEntity= new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
multipartEntity.addPart("uploaded", new FileBody(new File("myimage")));

Generally you access the files through getResources().openRawResource(R.id._your_id). If you absolutely need a File reference to it, one option is to copy it over to internal storage:
File file = new File(this.getFilesDir() + File.separator + "DefaultProperties.xml");
try {
InputStream inputStream = resources.openRawResource(R.id._your_id);
FileOutputStream fileOutputStream = new FileOutputStream(file);
byte buf[]=new byte[1024];
int len;
while((len=inputStream.read(buf))>0) {
fileOutputStream.write(buf,0,len);
}
fileOutputStream.close();
inputStream.close();
} catch (IOException e1) {}
Now you have a File that you can access anywhere you need it.

here are 2 functions. one to read from RAW and one from the Assets
/**
* Method to read in a text file placed in the res/raw directory of the
* application. The method reads in all lines of the file sequentially.
*/
public static void readRaw(Context ctx,int res_id) {
InputStream is = ctx.getResources().openRawResource(res_id);
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr, 8192); // 2nd arg is buffer
// size
// More efficient (less readable) implementation of above is the
// composite expression
/*
* BufferedReader br = new BufferedReader(new InputStreamReader(
* this.getResources().openRawResource(R.raw.textfile)), 8192);
*/
try {
String test;
while (true) {
test = br.readLine();
// readLine() returns null if no more lines in the file
if (test == null)
break;
}
isr.close();
is.close();
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
and from Assets folder
/**
* Read a file from assets
*
* #return the string from assets
*/
public static String getQuestions(Context ctx,String file_name) {
AssetManager assetManager = ctx.getAssets();
ByteArrayOutputStream outputStream = null;
InputStream inputStream = null;
try {
inputStream = assetManager.open(file_name);
outputStream = new ByteArrayOutputStream();
byte buf[] = new byte[1024];
int len;
try {
while ((len = inputStream.read(buf)) != -1) {
outputStream.write(buf, 0, len);
}
outputStream.close();
inputStream.close();
} catch (IOException e) {
}
} catch (IOException e) {
}
return outputStream.toString();
}

You can open it as InputStream, I don't know if possible as a file:
int rid = resources.getIdentifier(packageName + ":raw/" + fileName, null, null);
//get the file as a stream
InputStrea is = resources.openRawResource(rid);

You can use InputStreamBody instead of FileBody so you can use it like this:
InputStream inputStream = resources.openRawResource(R.raw.yourresource);
MultipartEntity multipartEntity= new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
multipartEntity.addPart("uploaded", new InputStreamBody(inputStream));