I'm trying to stream an image in the form of a byte[] to a Google App Engine servlet. I've done this before with servlets running on Tomcat, but for some reason doing so with GAE seems to be more problematic.
The byte array is being streamed fine from the client side and has the correct size, but it is always empty when being read on the server side.
Here's the important snippet of code doing the streaming from the client:
URL myURL = new URL("http://myapp.appspot.com/SetAvatar?memberId=1");
URLConnection servletConnection = myURL.openConnection();
servletConnection.setRequestProperty("Content-Type", "application/octet-stream");
servletConnection.setDoOutput(true);
servletConnection.setDoInput(true);
OutputStream os = servletConnection.getOutputStream();
InputStream is = servletConnection.getInputStream();
IOUtils.write(imageBytes, os);
os.flush();
os.close();
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();
Here's the code from the GAE servlet:
public void doPost(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
BufferedReader reader = request.getReader();
byte[] imageBytes = IOUtils.toByteArray(reader);
PrintWriter outputWriter = response.getWriter();
int len = request.getContentLength();
outputWriter.println("Content type is: " + request.getContentType());
outputWriter.println("Content length is: " + request.getContentLength());
outputWriter.println("Bytes read: " + imageBytes.length);
outputWriter.close();
}
The output from the server is:
Content type is: application/octet-stream
Content length is: 0
Bytes read: 0
I've tried just about everything like different readers and streams, but always with the same result: An empty byte array on the server side. I'm using the IOUtils class from the Apache Commons IO package, but I've tried without as well.
Any ideas why this is happening? Thanks in advance for any clues!
Related
I am new to XML-RPC and may be my question is silly but I can't find any information to help me for that...
So here it is : I am using this java code to send a XML file through a XML-RPC request using HTTP.
public static void sendXML(String file, String host, String port, String url) throws IOException{
Socket socket = new Socket(host, Integer.parseInt(port));
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
StringBuffer header = new StringBuffer();
header.append("POST "+url+" HTTP/1.0\n");
header.append("Content-Type: text/xml\n");
header.append("Content-Length: "+(new File(file).length()+2)+"\n");
header.append("\n");
byte[] buffer = header.toString().getBytes("UTF-8");
out.write(buffer);
InputStream src = new FileInputStream(file);
buffer = new byte[1024];
int b = 0;
while((b = src.read(buffer)) >= 0){
out.write(buffer, 0, b);
}
buffer = "\n\n".getBytes("UTF-8");
out.write(buffer);
out.close();
src.close();
in.close();
out.flush();
socket.close();
}
In this code, the XML file is already created, containing the method called and all the parameters.
This solution works fine but I am asking to make it compatible for a HTTPS protocol.
Do I need to only change the line
header.append("POST "+url+" HTTP/1.0\n");
in
header.append("POST "+url+" HTTPS/1.0\n");
?
Or should I use the Apache library https://ws.apache.org/xmlrpc/client.html ?
Or may be is there any simpler solution in java language ?
Thank you all for your help
For my project i need to download a pdf file from google drive using java
I get my httpresponse code 200 and by using following method i store it in abc.pdf file
String url = "https://docs.google.com/uc?id="+fileid+"&export=download";
URL obj = new URL(url);
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
// optional default is GET
conn.setRequestMethod("GET");
//add request header
conn.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
OutputStream f0 = new FileOutputStream("C:\\Users\\Darshil\\Desktop\\abc.pdf",true);
while ((inputLine = in.readLine()) != null) {
//System.out.println(inputLine);
byte b[]=inputLine.getBytes();
//System.out.println(b);
f0.write(b);
}
in.close();
f0.close();
But when i try to open abc.pdf in my adobe reader x i get following error:
There was an error opening this document.The file is damaged and could not be repaired
You seem to be directly accessing the Google drive using Raw HTTP requests.
You may be better of using the Google Drive SDK. This link contains good examples to address the use cases you state in your question.
However if you do want to stick to your technique then you should not be using a BufferedReader.readLine(). This is because the PDF file is a binary finally that would depend upon the correct byte sequences to be preserved in order to be read correctly by the PDF reader software. Hopefully the below technique should help you:
//read in chunks of 2KB
byte[] buffer = new byte[2048];
int bytesRead = 0;
try(InputStream is = conn.getInputStream())
{
try(DataOutputStream os = new DataOutputStream(new FileOutputStream("file.pdf"))
{
while((bytesRead = is.read(buffer)) != -1)
{
os.write(buffer, 0, bytesRead);
}
}
}
catch(Exception ex)
{
//handle exception
}
Note that I am using the try-with-resources statement in Java 7
Hope this helps.
I am teaching myself more about HTTP requests and such, so I wrote a simple POST request using Java's HttpURLConnection class and it returns compressed data which is easily decompress. I decided to go a lower level and send the HTTP request with sockets (for practice). I figured it out after a series of google searches, but there is one issue. When the server respondes with compressed data it isn't valid. Here is an image of a bit of debugging.
http://i.imgur.com/KfAcero.png
The portion below the "=" separator line is the response when using a HttpURLConnection instance, but the portion above it is the response when using sockets. I'm not too sure what is going on here. The bottom part is valid, while the top is not.
The HttpParameter and header classes simply store a key and value.
public String sendPost(String host, String path, List<HttpParameter> parameters, List<HttpHeader> headers) throws UnknownHostException, IOException {
String data = this.encodeParameters(parameters);
Socket socket = new Socket(host, 80);
PrintWriter writer = new PrintWriter(socket.getOutputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
writer.println("POST " + path + " HTTP/1.1");
for(HttpHeader header : headers) {
writer.println(header.getField() + ": " + header.getValue());
}
writer.println();
writer.println(data);
writer.flush();
StringBuilder contentBuilder = new StringBuilder();
for(String line; (line = reader.readLine()) != null;) {
contentBuilder.append(line + "\n");
}
reader.close();
writer.close();
return contentBuilder.toString();
}
Your problem is that you are using Readers and Writers for something that is not text.
InputStream and OutputStream work with bytes; Reader and Writer work with encoded text. If you try to use Reader and Writer with something that is not encoded text, you will mangle it.
Sending the request with a Writer is fine.
You want to do something like this instead:
InputStream in = socket.getInputStream();
// ...
ByteArrayOutputStream contentBuilder = new ByteArrayOutputStream();
byte[] buffer = new byte[32768]; // the size of this doesn't matter too much
int num_read;
while(true) {
num_read = in.read(buffer);
if(num_read < 0)
break;
contentBuilder.write(buffer, 0, num_read);
}
in.close();
writer.close();
return contentBuilder.toByteArray();
and make sendPost return a byte array.
I have this issue with GZIP compression:
I need to send by POST method a huge JSON string, which is too big to be accept like URL (Ex: http://localhost/app/send/JSON STRING ENCODED BY BASE64), than it result in HTTP error 403
so, I need to compress my json and I found a way to do it with GZIP compression, which I can decompress with gzdecode() in PHP.
but it doesn't work...
my functions compress() and decompress() works fine inside my Java App, but when I send it to webservice, something goes wrong and gzdecode() doesn't work.
I have no idea what I missing, I need some help
functions used in java app (client)
public String Post(){
String retorno = "";
String u = compress(getInput());
u = URLEncoder.encode(URLEncoder.encode(u, "UTF-8"));
URL uri = new URL(url + u);
HttpURLConnection conn = (HttpURLConnection) uri.openConnection();
conn.setDoOutput(false);
conn.setRequestMethod(getMethod());
conn.setRequestProperty("Content-encoding", "gzip");
conn.setRequestProperty("Content-type", "application/octet-stream");
BufferedReader buffer = new BufferedReader(
new InputStreamReader((conn.getInputStream())));
String r = "";
while ((r = buffer.readLine()) != null) {
retorno = r + "\n";
}
return retorno;
}
GZIP compress function (client)
public static String compress(String str) throws IOException {
byte[] blockcopy = ByteBuffer
.allocate(4)
.order(java.nio.ByteOrder.LITTLE_ENDIAN)
.putInt(str.length())
.array();
ByteArrayOutputStream os = new ByteArrayOutputStream(str.length());
GZIPOutputStream gos = new GZIPOutputStream(os);
gos.write(str.getBytes());
gos.close();
os.close();
byte[] compressed = new byte[4 + os.toByteArray().length];
System.arraycopy(blockcopy, 0, compressed, 0, 4);
System.arraycopy(os.toByteArray(), 0, compressed, 4,
os.toByteArray().length);
return Base64.encode(compressed);
}
method php used to receive a URL (server, using Slim/PHP Framework)
init::$app->post('/enviar/:obj/', function( $obj ) {
$dec = base64_decode(urldecode( $obj ));//decode url and decode base64 tostring
$dec = gzdecode($dec);//here is my problem, gzdecode() doesn't work
}
post method
public Sender() throws JSONException {
//
url = "http://192.168.0.25/api/index.php/enviar/";
method = "POST";
output = true;
//
}
As noticed in some of the comments.
Bigger data should be send as a POST request instead of GET. URL params should be used only for single variables. As you noticed the URL length is limited to few kB and it's not very good idea to send larger data this way (even though GZIP compressed).
Your GZIP compression code seems to be wrong. Please try this:
public static String compress(String str) throws IOException {
ByteArrayOutputStream os = new ByteArrayOutputStream(str.length());
GZIPOutputStream gos = new GZIPOutputStream(os);
gos.write(str.getBytes());
os.close();
gos.close();
return Base64.encodeToString(os.toByteArray(),Base64.DEFAULT);
}
i'm stuck on this process from two days, before posting i've searched a lot of topic and looks like it's a so simple issue. But i didn't get the problem.
Scenario is basic: i want to parse an XML from a remote computer through http connection:
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
try {
URL url = new URL("http://host:port/file.xml");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept","application/xml");
InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
PrintWriter pw = new PrintWriter("localfile_pw.xml");
FileOutputStream fos = new FileOutputStream("localfile_os.xml");
Then i tried three different ways to read the XML
Reading byte stream
byte[] buffer = new byte[4 * 1024];
int byteRead;
while((byteRead= is.read(buffer)) != -1){
fos.write(buffer, 0, byteRead);
}
Reading charachter per character
char c;
while((c = (char)br.read()) != -1){
pw.print(c);
System.out.print(c);
}
Reading line per line
String line = null;
while((line = br.readLine()) != null){
pw.println(line);
System.out.println(line);
}
In all cases my xml reading stops at the same point, after the same exact nuumber of bytes. And gets stuck without reading and without giving any exception.
Thanks in advance.
How about this (see IOUtils from Apache):
URL url = new URL("http://host:port/file.xml");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept","application/xml");
InputStream is = connection.getInputStream();
FileOutputStream fos = new FileOutputStream("localfile_os.xml");
IOUtils.copy(is, fos);
is.close();
fos.close();
The class supports persistent HTTP connections by default. If the size of the response is know at the time of the response, after it sends your data, the server will wait for another request. There are 2 ways of handling this:
Read the content-length:
InputStream is = connection.getInputStream();
String contLen = connection.getHeaderField("Content-length");
int numBytes = Integer.parse(contLen);
Read numBytes bytes from the input stream. Note: contLen may be null, in this case you should read until EOF.
Disable connection keep alive:
connection.setRequestProperty("Connection","close");
InputStream is = connection.getInputStream();
After sending the last byte of data the server will close the connection.