Develop Reference

How to download a file that gets processed over an HTTP request in Java?

I'm writing a program that builds stuff in a GUI (blah blah blah... irrelevant details), and the user is allowed to export that data as a .tex file which can be compiled to a PDF. Since I don't really want to assume they have a TeX environment installed, I'm using an API (latexonline.cc). That way, I can construct an HTTP GET request, send it to the API, then (hopefully!) return the PDF in a byte-stream. The issue, though, is that when I submit the request, I'm only getting the page data back from the request instead of the data from the PDF. I'm not sure if it's because of how I'm doing my request or not...
Here's the code:
... // preceding code
DataOutputStream dos = new DataOutputStream(new FileOutputStream("test.pdf"));
StringBuilder httpTex = new StringBuilder();
httpTex.append(this.getTexCode(...)); // This appends the TeX code (nothing wrong here)
// Build the URL and HTTP request.
String texURL = "https://latexonline.cc/compile?text=";
String paramURL = URLEncoder.encode(httpTex.toString(), "UTF-8");
URL url = new URL(texURL + paramURL);
byte[] buffer = new byte[1024];
try {
InputStream is = url.openStream();
int bufferLen = -1;
while ((bufferLen = is.read(buffer)) > -1) {
this.getOutputStream().write(buffer, 0, bufferLen);
}
dos.close();
is.close();
} catch (IOException ex) {
ex.printStackTrace();
}
Edit: Here's the data I'm getting from the GET request:
https://pastebin.com/qYtGXUsd

Solved! I used a different API and it works perfectly.
https://github.com/YtoTech/latex-on-http

Who is adding "\n" in Base64 encoded image when I write it in a file? Java

WHAT I'M DOING
I need to send via HTTPS request a JsonArray with some data and images in Base64 encoded strings. This works well if data is stored in memory.
Now, I need to avoid load all data in memory and I'm creating a temporally file in android device with all data that I need to send.
To create the file, I'm writting lots of JsonObjects inside him. Some of this JsonObjects have a field that represents the image. When I detect one, I get the image path and I encode it with Base64 as a String.
UPDATE:
First of all, I inicialize the file and I get the bufferedWriter
File f = new File(privateSincronizePath+File.separator+"upload_"+timefile+".json");
f.createNewFile();
FileWriter fw = new FileWriter(f.getAbsoluteFile(), true);
BufferedWriter bw = new BufferedWriter(fw);
Here is the code that create the image when exists:
JSONObject jobInf = new JSONObject();
jobInf.put("xx", "INSERT");
jobInf.put("xx", c.getString(c.getColumnIndex("xx")));
jobInf.put("xx", ""+c.getInt(c.getColumnIndex("xx")));
jobInf.put("xx", c.getString(c.getColumnIndex("xx")));
JSONObject jsonObject = new JSONObject(docu.filtre(dades, docu.getXx()));
Iterator<?> keys = jsonObject.keys();
boolean updated = false;
while(keys.hasNext() && !updated){
String key = (String)keys.next();
if (key != null && key.equals("path") && key.length() > 0){
jsonObject.put(key, getBase64EncodeFromImage(jsonObject.getString(key)));
updated = true;
}
}
jobInf.put("object", jsonObject);
escriure(bw, ","+jobInf.toString());
Method escriure():
UPDATE: This method is called every time I complete the creation of the JsonObject. Only append JsonObject as String to the file.
private void escriure(BufferedWriter bw, String s) throws IOException {
uploadLength += s.length();
bw.append(s);
}
Finally, when file is created, I'm reading it and, using OutputStream, I'm sending the data as Post parameters to the server:
this.Https_url = new URL(url+"/sync.php?");
HttpsURLConnection con = (HttpsURLConnection) Https_url.openConnection();
con.setReadTimeout(60000);
con.setConnectTimeout(60000);
con.setRequestMethod("POST");
con.setDoInput(true);
con.setDoOutput(true);
con.setFixedLengthStreamingMode(uploadLength);
OutputStream os = con.getOutputStream();
InputStream inp = new FileInputStream(new File(privateSincronizePath+File.separator+"upload_"+timefile+".json"));
byte[] buffer = new byte[1024];
int len;
while ((len = inp.read(buffer)) != -1) {
os.write(buffer, 0, len);
}
inp.close();
os.close();
// Establim la connexió:
con.connect();
WHAT IS THE PROBLEM
The problem is simple. When i open the image in the server, the file is corrupted and doesn't show the image.
WHAT I NOTICED
If I capture the image Base64 string encoded, before write in the file, and uploads it in the server, the image is Ok! Then, after write in the file, the image seems to be corrupted.
After investigate, I noticed that the Base64 encoded string after be written in the file, it have a lot of "\n" every x characters.
If I delete all of this "\n" or breaklines, the image can be opened by the server correctly.
THE QUESTION
Who is putting this breaklines? How can I write the Base64 encoded as String "as is" ?
Thanks all for your help in advance!
THE ANSWER
As Schoenobates answer, the solution was use the flag NO_WRAP.
To add more information, we put in the server side this function to read the encoded Base64 string with flag URL_SAFE
The function, obtained in on coment of TOM in php.net is:
<?php
function base64url_decode($base64url)
{
$base64 = strtr($base64url, '-_', '+/');
$plainText = base64_decode($base64);
return ($plainText);
}
?>
Thanks to StackOverflow, Axel and Schoenobates for your time!

That'll be the Android Base64 class - you need to set the flags to remove the newlines:
byte[] image = ...;
Base64.encodeToString(image, Base64.NO_WRAP | Base64.URL_SAFE);

My base64 encoded byte[] stream has extra characters after sent through a http response

I encode a pdf into a base64 byte[] stream and I want to send it as a http response to the browser. The problem is that the browser fails to load pdf.
I compared the base 64 string which I printed into the IDE console and the one from the browser console. The one from the IDE console is correct and the one from the browser has extra characters.
So, my base64 byte[]stream gets broken somehow when it's sent as a http response ? How do I solve this?
L.e. : The code
FileInputStream inputFileInputStream = null;
ServletOutputStream outputFileOutputStream = null;
File exportFile = new File(exportedReport);
int fileSize = (int) exportFile.length();
String fullPathToExport = exportFile.getAbsolutePath();
File fullPathFile = new File(fullPathToExport);
try {
// test to see if the path of the file is correct
System.out.println("The file is located at: "
+ exportFile.getAbsolutePath());
response.reset();
response.setContentType("application/pdf");
response.setContentLength(fileSize);
response.addHeader("Content-Transfer-Encoding", "base64");
response.setHeader( "Content-Disposition", "inline; filename=\"" + exportedReport +"\"");
inputFileInputStream = new FileInputStream(fullPathFile);
outputFileOutputStream = response.getOutputStream();
if (bytesToRead == -1) {
bytesToRead = (int)fullPathFile.length();
}
byte[] buffer = new byte[bytesToRead];
int bytesRead = -1;
while((inputFileInputStream != null) && ((bytesRead = inputFileInputStream.read(buffer)) != -1)){
if (codec.equals("base64")) {
//outputFileOutputStream.write(Base64.encodeBytes(buffer).getBytes("UTF-8"), 0, bytesToRead);
outputFileOutputStream.write(org.apache.commons.codec.binary.Base64.encodeBase64(buffer));
} else {
outputFileOutputStream.write(buffer, 0, bytesToRead);
}
}
inputFileInputStream.close();
outputFileOutputStream.flush();
outputFileOutputStream.close();

Your code has one major problem:
You are not sending one base64 encoded data part but many base64 encoded data parts (concatenated). But two or more base64 encoded data parts are not equal to one base64 encoded data part.
Example:
base64("This is a test") -> "VGhpcyBpcyBhIHRlc3Q="
base64("This ")+base64("is a ")+base64("test") -> "VGhpcyA=aXMgYSA=dGVzdA=="
You should use the org.apache.commons.codec.binary.Base64InputStream instead of the Base64.encodeBase64() utility method. Read the whole FileInputStream through it and you will get a valid base64 encoded data stream.
Anyway what you are doing is not necessary. You can transfer any 8 bit data via HTTP without further encoding.

posting data over network doesn't work in java.io API

I have the below piece of code that works fine when run from eclipse
String str = "testing";
InputStream is = new ByteArrayInputStream(str.getBytes());
int length = is.available();
byte[] data = new byte[length];
is.read(data, 0, length);
System.out.println("output "+new String(data));
When I break this code to post the data over network using Jerser client API and try read it back on the server using Jersey I don't get the value that I posted. The client code looks like this
public static void main(String s[]) throws IOException {
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
String str = "testingthi";
InputStream is = new ByteArrayInputStream(str.getBytes());
WebResource webResource = client.resource("http://192.168.1.15:8090/JersySample/resources/stream/upload");
ClientResponse response = webResource.type(MediaType.APPLICATION_OCTET_STREAM).post(ClientResponse.class,is);
is.close();
}
The code on the server looks like this
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public Response upload( InputStream is) {
try{
int size = is.available();
byte data[] = new byte[size];
is.read(data,0,size);
System.out.println(" Posted data is "+ new String(data)+" length is "+size+" stream size "+is.available());
is.close();
}catch (Exception e) {
e.printStackTrace();
}
return Response.ok().entity("Done").build();
}
When I read the string using apache-commons-io API I do get the string value that I posted. Can someone explain why it doesn't work using the java.io API which was working in non-network situation?

You're making the usual mistake of assuming that read() fills the buffer. It isn't obliged to do that. See the Javadoc. It's only obliged to read at least one byte and return. The byte count, return -1, or throw an IOException.
You're also misusing available(). It doesn't return the total number of bytes in the stream, and the way you're using it is specifically warned against in the Javadoc.

Java : Json exception

I am working in J2ME.
I am getting json response from server then I try to parse this json data as per my requirement.
this is my code for getting response from server:-
InputStream inputStream = null;
OutputStreamWriter out = null;
byte[] readData = new byte[50000];
String response = "no";
try {
// --- write ---
out = new OutputStreamWriter(connection.openOutputStream(), "UTF-8");
out.write(data.toString());
out.close();
// --- read ---
int responseCode = connection.getResponseCode();
if (responseCode != HttpConnection.HTTP_OK) {
throw new IOException("HTTP response code: " + responseCode);
}
inputStream = connection.openInputStream();
int actual = inputStream.read(readData);
response = new String(readData, 0, actual, "UTF-8");
return response;
}
when response from server is small then it works fine, but if response is large then it get half of response and return to my another method. Please suggest me what should I do to get large amount of data into my readData and response variable.
Thank you in advance.

You will need to read all data before (there can be more data in the stream).
As you have noticed the call to InputStream.read doesn't guarantee you to fill your buffer (readData) it return the number of bytes it could read at the time. It just doesn't matter how big your buffer is on your side.
You will need to read re-call the InputStream.read method to check that you have all the data available in the stream. The read method will return -1 when no more data is available.
This is an example how you can do it:
....
ByteArrayOutputStream bos = new ByteArrayOutputStream();
int read;
byte[] tmp = new byte[1024];
while ((read = inputStream.read(tmp)) != -1)
bos.write(tmp, 0, read);
return new String(bos.toByteArray(), "UTF-8");
Also, when you are done with the connection you should call close on it so that the system knows that you don't need it anymore.

You nead to flush and close your input stream after reading!!!
inputStream.close();