i just want to upload a file using jersey rest service and Jquery ajax as client here is my code
1. HTML
<form action="rest/file/upload" method="post" enctype="multipart/form-data">
<p>
Select a file : <input type="file" name="file" />
</p>
<input type="submit" value="Upload It" />
</form>
2.Rest Service
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(#FormDataParam("file") InputStream stream) {
String uploadedFileLocation = "E:\\\\uploaded\\test.jpg";
//Session s = Session.getDefaultInstance(new Properties());
//InputStream is = new ByteArrayInputStream(<< String to parse >>);
//MimeMessage message = new MimeMessage(s, stream);
//Multipart multipart = (Multipart) message.getContent();
// save it
writeToFile(stream, uploadedFileLocation);
String output = "File uploaded to : " + uploadedFileLocation;
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
return Response.status(200).entity(output).build();
}
// save uploaded file to new location
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
byte[] image = IOUtils.toByteArray(uploadedInputStream);
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
IOUtils.write(image, out);
/*int read = 0;
byte[] bytes = new byte[1024];
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}*/
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
its working but stream includes this line also
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg"
Content-Type: image/pjpeg
how to remove this from inputstream?
need expertise answers
This string that you see is a kind of identifier added by the server to mark the start and end of the data uploaded in a form. If you dump the whole data in a text file, it will show something like this in your text file.
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: multipart/form-data; boundary=-----------------------------7dd3b827a0ddc
Content-Length: 29278
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="txt1"
Some Sample Text
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg" Content-Type: image/jpeg
(Binary data not shown)
-----------------------------7dd3b827a0ddc--
The value of the boundary i.e. -----------------------------7dd3b827a0ddc is a marker that multipart form data uses to identify the start and end of data for all fields in your overall upload.
I created this sample file for you assuming one file upload and a input text named txt1.
On the data file you can see the "boundary" in header and then the boundary being used to separate the two fields in the form data. Notice the extra "--" on the last boundary. That marks the end of the file.
You need to manually parse the data and extract all of the fields. The data between the markers where you have filename="Jellyfish.jpg" is the actual binary data uploaded for your image. When you extract that data (excluding the "Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg" Content-Type: image/jpeg") from between the two markers and save that data as "Jellyfish.jpg"; this will be your image.
Related
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.FUNCTION) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) {
There's only HttpRequestMessage parameter on the run() method in the Azure Functions(Java) spec.
I need to declare and use MultipartHttpServletRequest to fetch a file from the multipart/data request.
I'm trying but cannot see any way to cast HttpRequestMessag to MultipartHttpServletRequest.
Please give me some advice.
The HttpTrigger spec is : https://learn.microsoft.com/en-us/java/api/com.microsoft.azure.functions.annotation.httptrigger?view=azure-java-stable
----------------------- update -------------------------
The uploaded image is still corrupted.
The size is exaclty same as the original one, but it seems like this :
I will paste the entire code. Please review it.
Function Class source :
public class HttpTriggerJava {
private static final String storageConnectionString =
"DefaultEndpointsProtocol=http;" +
"AccountName=00000;" +
"AccountKey=00000";
#FunctionName("HttpTriggerJava")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.FUNCTION) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) throws Exception{
context.getLogger().info("Java HTTP trigger processed a request.");
CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectionString);
CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
CloudBlobContainer container = blobClient.getContainerReference("contents");
// here the "content-type" must be lower-case
String contentType = request.getHeaders().get("content-type"); // Get content-type header
String body = request.getBody().get(); // Get request body
String boundary = contentType.split(";")[1].split("=")[1]; // Get boundary from content-type header
int bufSize = 1024;
InputStream in = new ByteArrayInputStream(body.getBytes()); // Convert body to an input stream
MultipartStream multipartStream = new MultipartStream(in, boundary.getBytes(), bufSize, null); // Using MultipartStream to parse body input stream
boolean nextPart = multipartStream.skipPreamble();
while(nextPart) {
String header = multipartStream.readHeaders();
System.out.println("");
System.out.println("Headers:");
System.out.println(header);
System.out.println("Body:");
if (header.contains("Content-Type: image/")) {
int start = header.indexOf("filename=")+"filename=".length()+1;
int end = header.indexOf("\r\n")-1;
String filename = header.substring(start, end);
System.out.println(filename);
FileOutputStream fos = new FileOutputStream(filename);
multipartStream.readBodyData(fos);
File sourceFile = new File(filename);
CloudBlockBlob blob = container.getBlockBlobReference(filename);
blob.uploadFromFile(sourceFile.getAbsolutePath());
} else {
multipartStream.readBodyData(System.out);
}
System.out.println("");
nextPart = multipartStream.readBoundary();
}
return request.createResponseBuilder(HttpStatus.OK).body("Success").build();
}
}
And the HTML is :
<head>
<script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
<script>
$(document).ready(function () {
$("#myFile").change(function() {
readURL(this);
});
$("#submit").click(function (event) {
event.preventDefault();
var form = $('#form')[0];
var data = new FormData(form);
$("#submit").prop("disabled", true);
$.ajax({
type: "POST",
enctype: 'multipart/form-data',
url: $(form).attr('action'),
data: data,
processData: false,
contentType: false,
cache: false,
timeout: 600000,
success: function (data) {
$("#result").text(data);
console.log("SUCCESS : ", data);
$("#submit").prop("disabled", false);
},
error: function (e) {
$("#result").text(e.responseText);
console.log("ERROR : ", e);
$("#submit").prop("disabled", false);
}
});
});
});
function readURL(input) {
if (input.files && input.files[0]) {
var reader = new FileReader();
reader.onload = function(e) {
$('#blah').attr('src', e.target.result).show();
}
reader.readAsDataURL(input.files[0]);
}
}
</script>
</head>
<body>
<form id=form
action="http://doopediafunctiontest.azurewebsites.net/api/HttpTriggerJava?code=00000"
method="post" enctype="multipart/form-data">
<p>
<br /> <br /> <strong>My file:</strong><br /> <input type="file" id="myFile" name="myFile">
<br /><img id="blah" src="#" alt="your image" style="display:none" />
</p>
<input id=submit type="submit" value="upload to Blob Storage">
</form>
<div id=result></div>
</body>
I compare the original image and the corrupted image by a hex editor.
And I found some random hexes changed to 3f, it should be the reason. Maybe there's some encoding problem. But how can I fix this?
(Please click to enlarge)
It sounds like you want to upload a file to your Azure Function with Http Trigger in Java via a HTML form with multipart/form-data like below.
<form method="POST" enctype="multipart/form-data" action="https://<your function app>/api/HttpTrigger-Java">
File to upload: <input type="file" name="upfile"><br/>
Notes about the file: <input type="text" name="note"><br/>
<br/>
<input type="submit" value="Press"> to upload the file!
</form>
However, there is not any class implements the interface HttpRequestMessage<T> and seems to not cast HttpRequestMessage to HttpServletRequest after I researched the source code of GitHub Repo Azure/azure-functions-java-library.
Per my experience, the only way is to parse the header and body of a multipart/form-data request to get the file. There is an answer of the similar SO thread Library and examples of parsing multipart/form-data from inputstream posted by the question owner, which includes the code using MultipartStream class of Apache Commons FileUpload that works after I test it.
Here is the Content-Type header and body of a multipart/form-data request received from Azure Function for Java.
Header Content-Type
content-type: multipart/form-data; boundary=----WebKitFormBoundaryT2TWuevX3RIYWRQF
multipart/form-data request body
------WebKitFormBoundaryT2TWuevX3RIYWRQF
Content-Disposition: form-data; name="upfile"; filename="z.txt"
Content-Type: text/plain
1234
ABCD
------WebKitFormBoundaryT2TWuevX3RIYWRQF
Content-Disposition: form-data; name="note"
test.txt
------WebKitFormBoundaryT2TWuevX3RIYWRQF--
Here is my sample code to fetch the file.
#FunctionName("HttpTrigger-Java")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) {
String contentType = request.getHeaders().get("content-type"); // Get content-type header
// here the "content-type" must be lower-case
String body = request.getBody().get(); // Get request body
InputStream in = new ByteArrayInputStream(body.getBytes()); // Convert body to an input stream
String boundary = contentType.split(";")[1].split("=")[1]; // Get boundary from content-type header
int bufSize = 1024;
MultipartStream multipartStream = new MultipartStream(in, boundary.getBytes(), bufSize, null); // Using MultipartStream to parse body input stream
// the code below comes from the SO thread above
// you can fetch a file content from readBodyData
// after the headers Content-Disposition: form-data; name="upfile"; filename="test.txt" \n Content-Type: text/plain
boolean nextPart = multipartStream.skipPreamble();
while (nextPart) {
String header = multipartStream.readHeaders();
System.out.println("");
System.out.println("Headers:");
System.out.println(header);
System.out.println("Body:");
multipartStream.readBodyData(System.out);
System.out.println("");
nextPart = multipartStream.readBoundary();
}
return request.createResponseBuilder(HttpStatus.OK).body("Success").build();
}
The output of code above in terminal :
Headers:
Content-Disposition: form-data; name="upfile"; filename="test.txt"
Content-Type: text/plain
Body:
1234
ABCD
Headers:
Content-Disposition: form-data; name="note"
Body:
test.txt
Update: If upload an image, the output of the code above is like below.
Headers:
Content-Disposition: form-data; name="upfile"; filename="test.jpg"
Content-Type: image/png
Body:
<the binary content of an image>
So you can parse the header to get the filename value to use FileOutputStream to store it, as the code below.
while(nextPart) {
String header = multipartStream.readHeaders();
System.out.println("");
System.out.println("Headers:");
System.out.println(header);
System.out.println("Body:");
if (header.contains("Content-Type: image/")) {
int start = header.indexOf("filename=")+"filename=".length()+1;
int end = header.indexOf("\r\n")-1;
String filename = header.substring(start, end);
System.out.println(filename);
FileOutputStream fos = new FileOutputStream(filename);
multipartStream.readBodyData(fos);
} else {
multipartStream.readBodyData(System.out);
}
System.out.println("");
nextPart = multipartStream.readBoundary();
}
Update 2:
I discovered there seems to be an issue of Azure Function for Java which may be a bug that will lose some bytes when uploading binary file, but it will not happend for uploading text file. So a workaround solution is to convert upload file to base64 string in browser to post to Azure Function and convert base64 content uploaded to the origin binary file in Azure Function.
Here is my testing HTML code.
File to upload: <input type="file" name="upfile" id="fileup"><br/>
<form method="POST" enctype="multipart/form-data" action="http://localhost:7071/api/HttpTrigger-Java">
Notes about the file: <input type="text" name="note"><br/>
<input type="hidden" name="file_base64" id="file_base64"><br/>
<input type="submit" value="Press"> to upload the file!
</form>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js">/script>
<script>
$(document).ready(function(){
$("#fileup").change(function(){
var v = $(this).val();
var reader = new FileReader();
reader.readAsDataURL(this.files[0]);
reader.onload = function(e){
console.log(e.target.result);
$('#file_base64').val(e.target.result);
};
});
});
</script>
The form above will post the header and body of base64 file chunk as below.
Header:
Content-Disposition: form-data; name="file_base64"
Body:
data:image/jpg;base64,iVBORw0KGgoAAAANSUhEUgAAB.............
My Java code in Azure Function:
import java.io.ByteArrayOutputStream;
import java.util.Base64;
if (header.equals("Content-Disposition: form-data; name=\"file_base64\"")) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
multipartStream.readBodyData(baos);
String content = baos.toString();
// System.out.println(content);
int index = content.indexOf(",")+1; // Get the index of base64 string in data-uploaded string
byte[] imgBytes = Base64.getDecoder().decode(content.substring(index)); // convert image base64 string to image byte arrays
....
// To upload image byte array to Blob Storage
// You can get the upload image filename from the form input `note`, please notes the order of form input elements.
} else {
multipartStream.readBodyData(System.out);
}
Why not use
HttpRequestMessage<Optional<byte[]>> request
instead of
HttpRequestMessage<Optional<String>> request
Transforming the body to String first messes things up.
I have a java server using sockets. I have an html file which contains a form that can be used to upload a file and send it. How can the server download that file without blocking. The browser never closes the connection and it just hangs in there I have beeping using readLine() from the input stream but it blocks. is there any way around this?
I appreciate any help here.
Thanks
please clarify your requirement, it seems total chaos in your steps.
did your program run as client side or server side?
could you show your code? and show what blocked you?
It is much better if you can draw a working-flow of your program .
Thanks for your supplement, do you want to upload a files through HTTP(writing in socket way). For this requirement, you could check the link link for how HTTP works
[RFC 1867] (https://www.rfc-editor.org/rfc/rfc1867)
Below is the package of how socket way do:
Suppose the server supplies the following HTML:
<FORM ACTION="http://server.dom/cgi/handle"
ENCTYPE="multipart/form-data"
METHOD=POST>
What is your name? <INPUT TYPE=TEXT NAME=submitter>
What files are you sending? <INPUT TYPE=FILE NAME=pics>
</FORM>
and the user types "Joe Blow" in the name field, and selects a text
file "file1.txt" for the answer to 'What files are you sending?'
The client might send back the following data:
Content-type: multipart/form-data, boundary=AaB03x
--AaB03x
content-disposition: form-data; name="field1"
Joe Blow
--AaB03x
content-disposition: form-data; name="pics"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
If the user also indicated an image file "file2.gif" for the answer
to 'What files are you sending?', the client might client might send
back the following data:
Content-type: multipart/form-data, boundary=AaB03x
--AaB03x
content-disposition: form-data; name="field1"
Joe Blow
--AaB03x
content-disposition: form-data; name="pics"
Content-type: multipart/mixed, boundary=BbC04y
--BbC04y
Content-disposition: attachment; filename="file1.txt"
Nebel & Masinter Experimental [Page 9]
RFC 1867 Form-based File Upload in HTML November 1995
Content-Type: text/plain
... contents of file1.txt ...
--BbC04y
Content-disposition: attachment; filename="file2.gif"
Content-type: image/gif
Content-Transfer-Encoding: binary
...contents of file2.gif...
--BbC04y--
--AaB03x--
I have server running on port. I want it to gather the data sent by a POST request which carries a file (Not just text but png image).
for example here is a server:
public void run() {
try {
InputStream is = insocket.getInputStream();
PrintWriter out = new PrintWriter(insocket.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String line;
line = in.readLine();
String request_method = line;
System.out.println("HTTP-HEADER: " + line);
line = "";
// looks for post data
int postDataI = -1;
while ((line = in.readLine()) != null && (line.length() != 0)) {
System.out.println("HTTP-HEADER: " + line);
if (line.indexOf("Content-Length:") > -1) {
postDataI = new Integer(
line.substring(
line.indexOf("Content-Length:") + 16,
line.length())).intValue();
}
}
String postData = "";
// read the post data
if (postDataI > 0) {
char[] charArray = new char[postDataI];
in.read(charArray, 0, postDataI);
postData = new String(charArray);
}
out.println("HTTP/1.0 200 OK");
out.println("Content-Type: text/html; charset=utf-8");
out.println("Server: MINISERVER");
// this blank line signals the end of the headers
out.println("");
// Send the HTML page
out.println("<H1>Welcome to the Mini Server</H1>");
out.println("<H2>Request Method->" + request_method + "</H2>");
out.println("<H2>Post->" + postData + "</H2>");
out.println("<form name=\"input\" action=\"form_submited\" method=\"post\">");
out.println("Username: <input type=\"text\" name=\"user\"><input type=\"submit\" value=\"Submit\"></form>");
out.close();
insocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
How can it be changed to safe a file (Uploaded through HTTP/1.1 POST REQUEST) form) to disk using this server.
Thank you for your time
I am using org.apache.http.HttpEntity for doing a multipart/form data POST to HTTPURLConnection to upload a file.
Here is the code that I am using.
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
String part1 = "\n{\"name\":\"test.txt\",\"creationTime\":1527023510389,\"fileUri\":\"/storage/test.txt\"}";
File file = new File("/storage/test.txt");
HttpEntity entity = MultipartEntityBuilder.create()
.setMode(HttpMultipartMode.BROWSER_COMPATIBLE)
.addBinaryBody("data", part1.getBytes(), ContentType.APPLICATION_JSON, "data.txt")
.addBinaryBody("file", file, ContentType.TEXT_PLAIN, filename)
.setBoundary(boundaryString)
.build();
OutputStream os = conn.getOutputStream();
entity.writeTo(os);
I see that the body is being posted as the following.
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
The problem is that the server requires a new line between the Content-Type and the contents of the first part. I've tried adding extra "\n" to the beginning contents (as seen but it gets erased when using HttpEntity.writeto().
The output that I want is the following:
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
I attempted to modify rewriting the output but not sure if this is the best way to do it by storing in a temporary file. The files I will be working with will be up to 20mb if that makes any difference.
entity.writeTo(new FileOutputStream("file.tmp"));
BufferedReader reader = new BufferedReader(new FileReader("file.tmp"));
OutputStream os = conn.getOutputStream();
PrintWriter writer = new PrintWriter(new BufferedOutputStream(os));
String str;
while ((str = reader.readLine()) != null) {
writer.println(str);
if (str.contains("Content-Type: ")) {
writer.println("\n");
}
}
writer.close();
reader.close();
os.close();
conn.connect();
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
// It's failing when accessing the above method
}
I tried running the above code and I get the following error:
java.lang.IllegalStateException: state: 2
at com.android.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:234)
at com.android.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:104)
at com.android.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:1156)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:976)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:509)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:438)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:567)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java)
It turns out that the HttpEntity.writeTo method is putting the necessary new lines but when I was printing the output to System.out, Android Studio's Logcat does not show plain new lines. I confirmed this by opening the file.tmp I was creating above and it had the proper new lines in there. It looks like there's some other error with the request since the body is valid for the server.
EDIT: Found the error in my request. I wasn't setting the Content-Type (I think I erased it while deleting some other code). I ended up using this to set the content type.
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());
A little history before my question, I posted a question about uploading multiple files to coldfusion using HTML5's multipart/form-data. And its worked beautifully.
Can you isolate code from being seen from CF10 compiler?
Our client has finally requested some unit tests for the RESTful functions that I have put together, and I've been able to get quite a lot done, but I've hit a roadblock with the massUpload function that I designed above.
Sorry for the long question, putting down what is relevant to the problem.
Here is the code in question:
Unit Test Code:
//Outside class calling sendHTTPrequest
HashMap<String,String> map = new HashMap<String,String>();
HashMap<String,File> getFiles = getFirstFileList();
map.put("testMethod", "massUploadTest");
map.put("method", "massUpload");
map.put("valueString1", valueString1);
map.put("valueString2", valueString2);
map.put("valueNumeric3", valueNumeric3);
map.put("valueBoolean4", valueBoolean4);
map.put("valueString5", valueString5);
map.put("valueBoolean6", valueBoolean6);
map.put("valueString7", valueString7);
try {
sendHTTPrequest(map, getFiles);
} catch(RuntimeException e) {
throw new RuntimeException("Fatal error in massUpload\n"
+ e.getMessage());
}
//End Call class code
Coldfusion function:
<cffunction name="massUpload" access="remote" returntype="string">
<cfargument name="valueString1" type="string" required="false">
<cfargument name="valueString2" type="string" required="false">
<cfargument name="valueNumeric3" type="numeric" required="false" default=0>
<cfargument name="valueBoolean4" type="boolean" required="true" default="false">
<cfargument name="valueString5" type="string" required="false">
<cfargument name="valueBoolean6" type="boolean" required="false" default="true">
<cfargument name="valueString7" type="string" required="true">
<!--- massUpload code --->
</cffunction>
So here is the code that is causing the major problems. I've tried different approaches to get the POST to work so obviously the code has problems. I'm trying to do this without downloading multiple software/libraries, but if there is no other way, then I'll make the necessary arrangements. Otherwise, would love to get this done with the standard Java library.
Function code:
//sendHTTPrequest method code
protected static final String BASE_URI = "<webaddress>/rest.cfc";
protected static final String CHARSET = "UTF-8";
protected String response;
protected int status;
protected String statusMessage;
protected void sendHTTPrequest(Map<String,String> map, Map<String, File> fileList) {
Set<String> keys = map.keySet();
status = 0;
response = null;
String boundary = "----" + System.currentTimeMillis();
try {
URL_CONNECTION = BASE_URL.openConnection();
HTTP_CONNECTION = (HttpURLConnection) (URL_CONNECTION);
//Set the request headers
HTTP_CONNECTION.setRequestMethod("POST");
URL_CONNECTION.setRequestProperty("Accept-Charset", CHARSET);
URL_CONNECTION.setRequestProperty("Content-type", "multipart/form-data; boundary=" + boundary);
//Set up a post request
URL_CONNECTION.setDoOutput(true);
OutputStream output = URL_CONNECTION.getOutputStream();
ByteArrayOutputStream bOutput = new ByteArrayOutputStream();
PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, CHARSET), true);
for(String key : keys) {
writer.write("--" + boundary);
writer.write(lineFeed);
writer.write("Content-Disposition: form-data; name=\"" + key + "\"");
writer.write(lineFeed);
writer.write(lineFeed);
writer.write(map.get(key));
writer.write(lineFeed);
}
FileInputStream inputStream;
for(Map.Entry<String, File> entry : fileList.entrySet()){
String name = entry.getKey();
writer.write("--" + boundary);
writer.write(lineFeed);
writer.write("Content-Disposition: form-data; "
+ "name=\"allfiles\"; filename=\"" + name + "\"");
writer.write(lineFeed);
String contentType = URLConnection.guessContentTypeFromName(name);
writer.write("Content-Type: " + contentType);
writer.write(lineFeed);
writer.write("Content-Transfer-Encoding: binary");
writer.write(lineFeed);
writer.write("Content-Id: <" + boundary + ">");
writer.write(lineFeed);
writer.write(lineFeed);
File temp = entry.getValue();
byte[] buffer = FileUtils.readFileToByteArray(temp);
output.write(buffer, 0, buffer.length);
output.flush();
writer.write(lineFeed);
}
writer.write("--" + boundary + "--");
writer.write(lineFeed);
writer.flush();
writer.close();
status = HTTP_CONNECTION.getResponseCode();
statusMessage = HTTP_CONNECTION.getResponseMessage();
if(status == SUCCESS) {
response = IOUtils.toString(URL_CONNECTION.getInputStream(), URL_CONNECTION.getContentEncoding());
}
System.out.println("Finished test " + map.get("testMethod") + " Status: " + status);
System.out.println("Response: " + response);
HTTP_CONNECTION.disconnect();
} catch(UnknownServiceException use) {
throw new RuntimeException("Protocol for the output is not supported");
} catch(IOException ioe) {
throw new RuntimeException("Unable to create the output stream");
}
}
//End sendHTTPrequest method code
I see in the output for status, response and statusMessage, that I get a 500 error, null, Internal Server Error.
Looking on the Coldfusion server I see:
SEVERE: Servlet.service() for servlet [CFCServlet] in context with path [/] threw exception
java.io.IOException: Corrupt form data: no leading boundary: %PDF-1.4 != ------1429222349902
at com.oreilly.servlet.multipart.MultipartParser.<init>(MultipartParser.java:182)
...
massUpload can handle different types of files, but is looking specifically for PDFs. So the unit test needs to be able to send different file types through to massUpload, not just PDFs.
Any insight into the problem would be welcome. Thank you.
(From comments)
Have you used a packet sniffer to see what the above actually generates?
Try flushing the writer just before you write the file bytes:
...
byte[] buffer = FileUtils.readFileToByteArray(temp);
writer.flush(); // flush stream here
output.write(buffer, 0, buffer.length);
...
Update:
Just to clarify, the reason for the original error is that even though the code creates a PrintWriter with autoFlush=true, it still does not automatically save text to the underlying OutputStream when write() is invoked. The autoFlush setting only affects these methods:
autoFlush - if true, the println, printf, or format methods
will [automatically] flush the output buffer
As a result, the code writes the file content to the stream before the boundary markers, which creates an invalid HTTP POST. Hence the 500 error. You can verify this using a package sniffer like Fiddler:
Original - (Invalid):
POST http://localhost:8888/test.cfm HTTP/1.1
Accept-Charset: UTF-8
Content-Type: multipart/form-data; boundary=14cd4c75e24
...
Content-Length: 65570
%PDF-1.4
%
2 0 obj <</Type/XObject/ColorSpace/DeviceGray/Subtype/Image/BitsPerComponent 8/Width 612/Length 11876/Height 792/Filter/FlateDecode>>stream
... more binary
--14cd4c75e24
Content-Disposition: form-data; name="allfiles"; filename="file0.pdf"
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-Id: <14cd4c75e24>
--14cd4c75e24--
The solution is to flush the writer before adding the file content. That ensures the boundary markers are added in the correct location, so the generated POST content is valid.
New Code (Valid)
POST http://localhost:8888/test.cfm HTTP/1.1
Accept-Charset: UTF-8
Content-Type: multipart/form-data; boundary=14cd4c91d29
...
Content-Length: 65570
--14cd4c91d29
Content-Disposition: form-data; name="allfiles"; filename="file0.pdf"
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-Id: <14cd4c91d29>
%PDF-1.4
%
2 0 obj <</Type/XObject/ColorSpace/DeviceGray/Subtype/Image/BitsPerComponent 8/Width 612/Length 11876/Height 792/Filter/FlateDecode>>stream
...
As an aside, I believe Content-Id must be unique. So "boundary" probably is not a good choice for the Content-Id value.
I just want to send a text file and a JPEG file over the network. fortunately, i have access to both the server code and the client code. Here's my (google app engine) code.
private void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename filename = new GcsFilename("my-bucket", "my-file");
Builder fileOptionsBuilder = new GcsFileOptions.Builder();
fileOptionsBuilder.mimeType("text/html");
GcsFileOptions fileOptions = fileOptionsBuilder.build();
GcsOutputChannel outputChannel = gcsService.createOrReplace(filename, fileOptions);
byte[] buffer = new byte[1024];
InputStream reader = req.getInputStream();
BufferedOutputStream outStream = new BufferedOutputStream(Channels.newOutputStream(outputChannel));
while(true) {
int bytesRead = reader.read(buffer);
if (bytesRead == -1) {
break; // have a break up with the loop.
} else if (bytesRead < 1024) {
byte[] temp = Arrays.copyOf(buffer, bytesRead);
outStream.write(temp);
} else {
outStream.write(buffer);
}
}
outStream.close();
outputChannel.close();
}
As you can see, i use a raw InputStream to get all the data that is sent over the net.
and on the client side, i send a text file over like so: (in Android)
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpost = new HttpPost("http://my-hosted-url/postit");
MultipartEntity entity = new entity.addPart("myImageFile", new FileBody(someLogFile));
httpost.setEntity(entity);
HttpResponse response;
response = httpClient.execute(httpost);
This works just fine... sort of. the problem is that when i try to view the file/data that is sent, it has a header on top of it, as such:
--NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myString" Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit STRING_VALUE ---NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myImageFile"; filename="something.txt" Content-Type: application/octet-stream Content-Transfer-Encoding: binary
[Thu Aug 14 17:14:26 PDT 2014] then the real log starts here...
How do i get rid of the headers that is somehow stuck to the body?
What you have here is a multipart request. It is a single request where the body consists of the various parts separated by a separator string.
In your case, it's more easily viewed as:
--NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myString"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
STRING_VALUE
---NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myImageFile"; filename="something.txt"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
[binary here]
It has two parts where each part has its corresponding headers and body. I'm guessing you're interested in the bodies. You'll need to extract them.
You can either read the HTTP specification and/or the specification about multipart requests and write your own parser, or you can use some built-in (I don't know if GAE is Servlet 3.0 ready or not) or 3rd party methods. See these
How can my Servlet receive parameters from a multipart/form-data form?
Convenient way to parse incoming multipart/form-data parameters in a Servlet
How can I handle multipart/form-data POST requests in my java servlet?