Develop Reference

Java - Obtain PDF from URL and return BASE64 string

I have the following task to obtain a PDF from URL and return a BASE64 string.
What I have currently (sorry I am not a Java Expert):
public String readPDFSOAP(String var, Container container) throws StreamTransformationException{
try {
//get the url page from the arguments array
URL url = new URL("URLPDF");
try {
//get input Stream from URL
InputStream in = new BufferedInputStream(url.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[131072];
int n = 0;
while (-1 != (n = in.read(buf))) {
out.write(buf, 0, n);
}
out.close();
in.close();
byte[] response = out.toByteArray();
String string = new String(response);
} catch (Exception e) {
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
}return String;}
But the string can't be returned.
Any help is appreciated.
Thanks,
Julian

Your code is all kinds of wrong. For starters, use the Base64 class to handle encoding your byte array. And no need to assign it to a variable, just return it.
return Base64.getEncoder().encodeToString(response)
and on your last line, outside of your try/catch block, just throw an exception. If you get there then you weren't able to properly retrieve and encoded the response, so no need to return a value. You're in an error condition.

Use java.util.Base64.
PDFs can be pretty large. Instead of reading it into memory, encode the InputStream directly:
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (InputStream in = new BufferedInputStream(url.openStream())) {
in.transferTo(Base64.getEncoder().wrap(out));
}
String base64 = out.toString(StandardCharsets.US_ASCII);
The Base64 encoded version is even larger than the original file. I don’t know what you plan to do with the encoded version, but if you’re planning to write it somewhere, you want to avoid keeping any version of the file—original or encoded—in memory. You can do that by having your method accept an OutputStream as an argument:
public void readPDFSOAP(OutputStream destination,
String var,
Container container)
throws StreamTransformationException,
IOException {
URL url = new URL("https://example.com/doc.pdf");
try (InputStream in = new BufferedInputStream(url.openStream())) {
in.transferTo(Base64.getEncoder().wrap(destination));
}
}
Update:
Since you have said you cannot use a try-with-resources statement:
A try-with-resources statement is just a convenient way to guarantee an InputStream (or other closeable resource) is closed. This:
try (InputStream in = new BufferedInputStream(url.openStream())) {
// code that uses 'in'
}
is (nearly) equivalent to this:
InputStream in = null;
try {
in = new BufferedInputStream(url.openStream());
// code that uses 'in'
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
// Suppress
}
}
}

Python generated zlib + base64 text getting incorrect header check exception in Java

I am required to write a consumer in Java from Kafka streams, publisher app is written by a third party in python. When I use base64 decoding and then zlip uncompress in java, I get an incorrect header check exception.
My Task is: Convert this compressed base64 + zlib data into a text file that is readable.
Publisher's code in python:
# read in the file content
inputf=open(file, 'rb')
input=inputf.read()
inputf.close()
# using zlib.compress(s) method
compressed = zlib.compress(input)
# encoding with base64 encoding
encoding_type='base64'
enc_data=encode(compressed,encoding_type)
enc_data_utf8=enc_data.decode("utf-8")
# enc_data=enc_data_no_no_newline ####[0:86000] # trim
event_etl_event[filename+"_content"]=enc_data_utf8
event_etl_event[filename+"_compressed_format"]="zlib+uuencode"
enter code here
Consumer Code in java
public void processData(){
inputStr = event.getEventEtlEvent().getAllProblemsTxtContent();
System.out.println("Before Base64 decoding: \n" + inputStr);
Path path0 = Paths.get("AllProblems_Before_base64_decoding.txt");
Files.write(path0, inputStr.getBytes());
Base64 base64 = new Base64();
String decodedString = new String(base64.decode(inputStr.getBytes()));
System.out.println("After Base64 decode: \n" + decodedString);
Path path1 = Paths.get("AllProblems_After_base64_decoding.txt");
Files.write(path1, decodedString.getBytes());
System.out.println("now zlib 64 decodingString .........\n\n\n");
byte[] output = ZLibUtils.decompress(decodedString.getBytes());
System.out.println("After Zlib Decompress: "+ output);
} catch (JsonParseException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} /*catch (InvalidDataException e) {
e.printStackTrace();
} catch (DataFormatException e) {
e.printStackTrace();
}*/catch (Exception e) {
e.printStackTrace();
}
ZLibUtils.java
public static byte[] decompress(byte[] data) throws DataFormatException {
byte[] output = new byte[0];
Inflater decompresser = new Inflater();
decompresser.reset();
decompresser.setInput(data);
ByteArrayOutputStream o = new ByteArrayOutputStream(data.length);
// try {
byte[] buf = new byte[data.length];
byte[] a = new byte[data.length];
while (!decompresser.finished()) {
int i = decompresser.inflate(buf);
o.write(buf, 0, i);
}
output = o.toByteArray();
/* } catch (Exception e) {
output = data;
e.printStackTrace();
//FIXME: in later code
System.exit(0);
} finally {
try {
o.close();
} catch (IOException e) {
e.printStackTrace();
}
} */
decompresser.end();
return output;
}
Now when I run my program, i get the following exception:
java.util.zip.DataFormatException: incorrect header check
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:259)
at java.util.zip.Inflater.inflate(Inflater.java:280)
at com.exmple.util.ZLibUtils.decompress(ZLibUtils.java:84
)
Looking forward to hearing from you soon.

The problem is that you convert the base64 decoded data from a byte array into a string and back into a byte array. For most encodings this is not a no-op. That means, for most encodings and most byte arrays,
byte[] decoded = { (byte) 0x9b, 1, 2, 3 };
String decodedString = new String(decoded);
byte[] processed = decodedString.getBytes();
the contents of processed will be different from the contents of decoded.
The solution is to not treat the base64 decoded data as string and instead of that work with the byte data directly:
Base64 base64 = new Base64();
byte[] decoded = base64.decode(inputStr.getBytes());
Path path1 = Paths.get("AllProblems_After_base64_decoding.txt");
Files.write(path1, decoded);
System.out.println("now zlib 64 decodingString .........\n\n\n");
byte[] output = ZLibUtils.decompress(decoded);

How to calculate the downloaded file size and total data to be downloaded

How to calculate the amount of data downloaded and the total data to be downloaded using socket.
E.G. 500kb/95000kb...95000kb/95000kb
Here i included my code for your reference.
private static void updateFile() {
Socket socket = null;
PrintWriter writer = null;
BufferedInputStream inStream = null;
BufferedOutputStream outStream = null;
try {
String serverName = System.getProperty("server.name");
socket = new Socket(serverName, 80);
writer = new PrintWriter(socket.getOutputStream(), true);
inStream = new BufferedInputStream(socket.getInputStream());
outStream = new BufferedOutputStream(new FileOutputStream(new File("XXX.txt")));
// send an HTTP request
System.out.println("Sending HTTP request to " + serverName);
writer.println("GET /server/source/path/XXX.txt HTTP/1.1");
writer.println("Host: " + serverName + ":80");
writer.println("Connection: Close");
writer.println();
writer.println();
// process response
int len = 0;
byte[] bBuf = new byte[8096];
int count = 0;
while ((len = inStream.read(bBuf)) > 0) {
outStream.write(bBuf, 0, len);
count += len;
}
}
catch (Exception e) {
System.out.println("Error in update(): " + e);
throw new RuntimeException(e.toString());
}
finally {
if (writer != null) {
writer.close();
}
if (outStream != null) {
try { outStream.flush(); outStream.close(); } catch (IOException ignored) {ignored.printStackTrace();}
}
if (inStream != null) {
try { inStream.close(); } catch (IOException ignored) {ignored.printStackTrace();}
}
if (socket != null) {
try { socket.close(); } catch (IOException ignored) {ignored.printStackTrace();}
}
}
}
Please advice to achieve this and thanks in advance.

The socket in general does not know the size of receiving data. A socket is bound to a TCP connection and TCP does not provide any information about data size. It is task for the application protocol and it is HTTP in your example.
HTTP indicates the data size in the Content-Length header. The HTTP response looks like:
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 13
Connection: keep-alive
<html></html>
The HTML response contains header and body. The body is separated from header by a newline. The Content-Length header contains the size of the body in bytes.
So you may either parse the header and find the length or use an existing class like java.net.HttpURLConnection.

Streaming encrypted audio bytes over socket

I have updated the question after the comments to make the problem clearer.I am trying to write a code for encrypted voice call over TCP. Of course SSL is one of the options when it comes to a secure connection, but just now I'm working on TCP, like any other project.
The program aims to capture the audio coming from microphone, encrypt it using AES and then send the data to the server.On the server side the received data would be decrypted and sent to the speaker. But with these codes I get a LineUnavailable Exception from the client side during runtime:
Unable to open the line: javax.sound.sampled.LineUnavailableException: line with format PCM_SIGNED 16000.0 Hz, 8 bit, stereo, 2 bytes/frame, not supported.
Normally, without encryption there is no problem with the codes, while I use a BufferedOutputStream(s.getOutputStream()) object for the recorded data; and the sound is transmitted over network successfully, i.e my hardware supports the mentioned PCM format.
On the other hand, in a simple standalone capture/play java module I have tested the encryption codes, where I have managed to encrypt and decrypt the audio data between the capturing and saving processes, while it was a byte[] data, just before it was saved to an output wav file. There seems no problem with the encryption process itself.
The problem arises during networking, when I use a ByteArrayOutputStream to write the audio data coming from DataLine, instead of using a BufferedOutputStream object which was directly taking the s.getOutputStream() method before. The reason to use a ByteArrayOutputStream here is to pass the captured bytes as an input argument to the encrypting method. It is worth to note that the ByteArrayOutputStream works well, while saving audio bytes to a wav file on local disk when there is no sockets around.
The question for me now is the disintegration between a working OutputStream object and the bytes waiting for encryption, considering a TCP network. My final test was the unsuccessful one with DataOutput/InputStream objects in the code below. Still need any ideas for an appropriate Input/Output streaming method to achieve a successful communication, if any.
The code piece for capturing and sending data is:
public void run() {
try {
dos = new DataOutputStream(s.getOutputStream());// need the exact stream obj.
} catch (IOException ex) {
return;
}
AudioFormat format =new AudioFormat(16000,8,2,true,true);
DataLine.Info info = new DataLine.Info(TargetDataLine.class,format);
if (!AudioSystem.isLineSupported(info)) {
System.err.println("Line matching " + info + " not supported.");// throws the exception
return;
}
try {
line = (TargetDataLine) AudioSystem.getLine(info);
line.open(format, line.getBufferSize());
} catch (LineUnavailableException ex) {
System.err.println("Unable to open the line: " + ex);// related to exception
return;
}
byte[] data = new byte[256];
int numBytesRead=0;
line.start();
// In the nonsecure call version, the audio data is written directly
// to the BufferedOutputStream(s.getOutputStream()) and transmitted without problem
ByteArrayOutputStream caps = new ByteArrayOutputStream(); //?
while (thread != null) {
numBytesRead = line.read(data, 0,128);
try {
caps.write(data, 0, numBytesRead);
} catch (Exception ex) {
System.err.println("Unable to read/write line data: " + ex);
break;
}
}
line.stop();
line.close();
line = null;
try {
caps.flush();
caps.close();
} catch (IOException ex) { ex.printStackTrace(); }
try {
SecretKeySpec skeySpec = CryptUtil.getSecretKeySpec("password12345678","AES",128);
byte[] encrypted = CryptUtil.encrypt(caps.toByteArray(), skeySpec);
dos.writeInt(encrypted.length);
dos.write(encrypted,0,encrypted.length);
} catch (Exception ex) { }
}
The code piece for receiving and playing data is:
public void run() {
AudioFormat format =new AudioFormat(16000,8,2,true,true);
try {
dis = new DataInputStream(s.getInputStream());// need the exact stream obj.
} catch (Exception e) {
System.err.println("Could not get InputStream: " + e);
}
try {
int length = dis.readInt();
byte[] message = new byte[length];
dis.readFully(message);
SecretKeySpec skeySpec = CryptUtil.getSecretKeySpec("password12345678","AES",128);
byte[] audioBytes = CryptUtil.decrypt(message, skeySpec);
ByteArrayInputStream bais = new ByteArrayInputStream(audioBytes);
playStream = new AudioInputStream(bais, format, audioBytes.length / format.getFrameSize());
} catch (Exception e) {
System.err.println("Could not decrypt the stream: " + e);
}
DataLine.Info info = new DataLine.Info(SourceDataLine.class,format);
try {
line = (SourceDataLine) AudioSystem.getLine(info);
line.open(format, bufSize);
} catch (LineUnavailableException ex) {
System.err.println("Unable to open the line: " + ex);
return;
}
byte[] data = new byte[256];
int numBytesRead = 0;
line.start();
while (thread != null) {
try{
numBytesRead = playStream.read(data);
line.write(data, 0,numBytesRead);
} catch (Exception e) {
System.err.println("Error during playback: " + e);
break;
}
}
if (thread != null) {
line.drain();
}
line.stop();
line.close();
line = null;
}

Best to try and use a simpler approach for leveraging streaming encryption in your code using Encryptor4j: https://github.com/martinwithaar/Encryptor4j
Wrap an OutputStream for encryption:
Encryptor encryptor = new Encryptor(secretKey, "AES/CTR/NoPadding", 16);
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream("original.jpg");
os = encryptor.wrapOutputStream(new FileOutputStream("encrypted.jpg"));
byte[] buffer = new byte[4096];
int nRead;
while((nRead = is.read(buffer)) != -1) {
os.write(buffer, 0, nRead);
}
os.flush();
} finally {
if(is != null) {
is.close();
}
if(os != null) {
os.close();
}
}
And an InputStream for decryption:
Encryptor encryptor = new Encryptor(secretKey, "AES/CTR/NoPadding", 16);
InputStream is = null;
OutputStream os = null;
try {
is = encryptor.wrapInputStream(new FileInputStream("encrypted.jpg"));
os = new FileOutputStream("decrypted.jpg");
byte[] buffer = new byte[4096];
int nRead;
while((nRead = is.read(buffer)) != -1) {
os.write(buffer, 0, nRead);
}
os.flush();
} finally {
if(is != null) {
is.close();
}
if(os != null) {
os.close();
}
}

OutOfMemory when creating Base64 string in java?

I used ostermillerutils library to create base64 string but I get OutOfMemory error if the image is heavy. If the image I try to convert is a simple image, the code is working fine.
public String createBase64String(InputStream in) {
//collect = new ByteArrayOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for(int readNum; (readNum = in.read(buf)) != -1; ) {
bos.write(buf, 0, readNum);
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
return null;
}
finally {
if (in != null) {
try {
in.close();
}
catch (IOException ex) {
;
}
}
}
byte[] ba = bos.toByteArray();
String coded = Base64.encodeToString(ba);
return coded;
}
I also tried doing this but the base64 was incorrect when I tried to decode it.
public void createBase64String(InputStream in) throws IOException {
//collect = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int readNum = 0;
try {
while((readNum = in.read(buf)) != -1)
{
smtp.addBase64(Base64.encodeBase64String(buf));
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
}
finally {
if (in != null) {
in.close();
}
}
}
Please suggest solutions for JDK 1.4 and also for later versions of Java.

If you like to write the encoded content straight into a file then use the following code
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
IOUtils class from Apache Commons IO.
EDIT
Since you want to do it using BufferedWriter, use it as follows
OutputStream out = Base64OutputStream(smtpSocket.getOutputStream());
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
IOUtils.copy(is, bw);

It sounds like the problem is that you're having to manipulate too much data in memory when you read the entire image. One fix would be to increase the Java heap size until you have enough memory, but that would just be avoiding the problem rather than solving it.
A better option would be to look at a streaming implementation of a Base64 encoder. This would mean you're only working on a subset of the image at any time. I believe that Base64OutputStream from Apache Commons would do the job for you.

I've fixed my problem by using javabase64-1.3.1.jar library.
OutputStream fos2 = FileUtil.getOutputStream(base64FileName, FileUtil.HDD);
InputStream in2 = FileUtil.getInputStream(fileName, FileUtil.HDD);
Base64.encode(in2, fos2);
in2.close();
fos2.close();
I stored the base64 string to a text file first.
public void createBase64String(InputStream in) throws IOException {
baos = new ByteArrayOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int readNum = 0;
smtp.addBase64("\t\t");
try {
while ((readNum = in.read(buf)) >= 0) {
baos.write(buf, 0, readNum);
smtp.addBase64(baos.toString());
baos.reset();
}
}
catch (IOException ex) {
LogUtil.error("Sending of Base64 String to SMTP: IOException: " + ex);
}
finally {
if (in != null) {
in.close();
baos.close();
}
}
baos = null;
buf = null;
}
then send each line to smtp's socket outputstream.

From Java 8 onwards, there is a simple way to implement base64 encoding in an output stream with one line of code and no external dependencies:
import java.util.Base64;
OutputStream os = ...
OutputStream base64 = Base64.getEncoder().wrap(os);
Base64 also provides other flavors of base64 encoder; see javadocs:
Base64
Base64.Encoder.wrap