A webservice returns a MIME file as InputStream with following content. I use Java Apache HTTPClient in order to make a request:
MIME-Version:1.0
Content-Type:multipart/mixed;
boundary="----=_Part_58_1750763977.1605815692305"
------=_Part_58_1750763977.1605815692305
Content-Type: application/octet-stream; name=preview.pdf
Content-ID: response-1
Content-Disposition: attachment; filename=preview.pdf
%PDF-1.7
[...]
------=_Part_67_626667127.1605818243111
Content-Type: application/octet-stream; name=thumbnail.jpg
Content-ID: response-2
Content-Disposition: attachment; filename=thumbnail.jpg
------=_Part_58_1750763977.1605815692305
Content-Type: application/octet-stream; name=report.xml
Content-ID: response-3
Content-Disposition: attachment; filename=report.xml
How can I now convert these chunks to single files? I tried javax.Mail and MIME file, but didn't succeed.
I managed to parse the message as mime and save the attachments as file:
HttpResponse response = client.execute(post);
Session s = Session.getInstance(new Properties());
MimeMessage myMessage = new MimeMessage(s, response.getEntity().getContent());
Multipart multipart = (Multipart) myMessage.getContent();
for (int i = 0; i < multipart.getCount(); i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
InputStream is = bodyPart.getInputStream();
File f = new File("tmp/" + bodyPart.getFileName());
FileOutputStream fos = new FileOutputStream(f);
byte[] buf = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buf))!=-1) {
fos.write(buf, 0, bytesRead);
}
fos.close();
}
by using
compile([group: 'tech.blueglacier', name: 'email-mime-parser', version: '1.0.5'])
Related
I'm trying to share from Microsoft's Excel Android app to my custom Android app that will take the shared excel workbook and send it in an email through Gmail's API.
After my app sends the email and I try to open the excel file on my computer, I get there error, "We found a problem with some content in 'May 25.xlsx'."
It does, however, open in the desktop Chrome browser using Google Sheets.
Here's the Uri of the workbook: content://com.microsoft.office.excel.excelApplication.provider/docsui_temp_share_files/fea009cf-65c3-46f4-8dda-50758298b9fc/May%2025.xlsx
Here's the code that turns it into a file.
Context ctx; //defined by Android
Uri uri; //comes in populated with the value above
String filename; //generated from the Uri in a separate method
InputStream is = ctx.getContentResolver().openInputStream(uri);
File directory = ctx.getCacheDir();
FileOutputStream fos = null;
File fileToReturn = null;
try {
final File tempFile = File.createTempFile(filename.split("\\.")[0], "." + filename.split("\\.")[1], directory);
tempFile.deleteOnExit();
fos = new FileOutputStream(tempFile);
byte[] bytes = new byte[1024 * 16];
int read = -1;
while ((read = is.read(bytes)) != -1) {
fos.write(bytes);
}
fileToReturn = tempFile;
}
finally {
if (fos != null) {
fos.close();
}
}
Here's the code that creates the MimeMessage to send. The attachments parameter has the File created from above.
private MimeMessage createEmail(List<String> toAddresses,
String from,
String subject,
String bodyText, ArrayList<File> attachments) throws MessagingException {
if(attachments == null){
attachments = new ArrayList<>();
}
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage email = new MimeMessage(session);
InternetAddress fAddress = new InternetAddress(from);
for(String toAddress : toAddresses) {
InternetAddress toInternetAddress = new InternetAddress(toAddress);
email.addRecipient(javax.mail.Message.RecipientType.TO, toInternetAddress );
}
email.setFrom(fAddress);
email.setSubject(subject);
Multipart multipart = new MimeMultipart();
BodyPart textBody = new MimeBodyPart();
textBody.setText(bodyText);
multipart.addBodyPart(textBody);
for (File attachment : attachments){
MimeBodyPart attachmentBody = new MimeBodyPart();
DataSource source = new FileDataSource(attachment.getAbsolutePath());
attachmentBody.setDataHandler(new DataHandler(source));
attachmentBody.setFileName(attachment.getName());
multipart.addBodyPart(attachmentBody);
}
email.setContent(multipart);
return email;
}
And the code to send the email.
private String sendMessage(Gmail service,
String userId,
MimeMessage email)
throws MessagingException, IOException {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
email.writeTo(bytes);
ByteArrayContent messageByteArrayContent = new ByteArrayContent("message/rfc822", bytes.toByteArray());
Message message;
message = service.users().messages().send(userId, null, messageByteArrayContent).execute();
}
Here's what comes through as the email.
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----=_Part_0_262386509.1596398610889"
Date: Sun, 2 Aug 2020 13:03:34 -0700
------=_Part_0_262386509.1596398610889
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
May 252265647717182285060.xlsx
------=_Part_0_262386509.1596398610889
Content-Type: application/octet-stream; name="May 25.xlsx"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="May 25.xlsx"
------=_Part_0_262386509.1596398610889--
Note that the Content-Type is application/octect-stream.
I've made this application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, by manually adding a Content-Type header, but to no avail.
This code works to send almost anything else. And if I save the excel file first and share it from my file manager app, it opens just fine.
And it seems specific to an excel file. If I share it as a pdf from the Excel app (that's an option), the pdf opens fine.
I also encounter the same error if I try to share from the Google Sheets app, and share it as an .xlsx file. But again, if I try to share it as anything else, like a .csv or .pdf, it opens fine on the other side.
Here's the email if it's shared as a .pdf file from the Google Sheets app. Again, the content type is application/octet-stream, but it opens fine here.
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----=_Part_4_203349844.1596399067869"
Date: Sun, 2 Aug 2020 13:11:10 -0700
------=_Part_4_203349844.1596399067869
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Untitled spreadsheet8410545067288443069.pdf
------=_Part_4_203349844.1596399067869
Content-Type: application/octet-stream; name="Untitled spreadsheet8410545067288443069.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Untitled spreadsheet8410545067288443069.pdf"
------=_Part_4_203349844.1596399067869--
In the code, where you are writing the xlsx file, you are always reading a specified buffer size (1024 * 16), and writing to file. I think on last read, if the bytes you read are less than the buffer size of your bytes array, you would be writing extra unnecessary bytes. This could be your file corruption reason.
I am using org.apache.http.HttpEntity for doing a multipart/form data POST to HTTPURLConnection to upload a file.
Here is the code that I am using.
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
String part1 = "\n{\"name\":\"test.txt\",\"creationTime\":1527023510389,\"fileUri\":\"/storage/test.txt\"}";
File file = new File("/storage/test.txt");
HttpEntity entity = MultipartEntityBuilder.create()
.setMode(HttpMultipartMode.BROWSER_COMPATIBLE)
.addBinaryBody("data", part1.getBytes(), ContentType.APPLICATION_JSON, "data.txt")
.addBinaryBody("file", file, ContentType.TEXT_PLAIN, filename)
.setBoundary(boundaryString)
.build();
OutputStream os = conn.getOutputStream();
entity.writeTo(os);
I see that the body is being posted as the following.
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
The problem is that the server requires a new line between the Content-Type and the contents of the first part. I've tried adding extra "\n" to the beginning contents (as seen but it gets erased when using HttpEntity.writeto().
The output that I want is the following:
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
I attempted to modify rewriting the output but not sure if this is the best way to do it by storing in a temporary file. The files I will be working with will be up to 20mb if that makes any difference.
entity.writeTo(new FileOutputStream("file.tmp"));
BufferedReader reader = new BufferedReader(new FileReader("file.tmp"));
OutputStream os = conn.getOutputStream();
PrintWriter writer = new PrintWriter(new BufferedOutputStream(os));
String str;
while ((str = reader.readLine()) != null) {
writer.println(str);
if (str.contains("Content-Type: ")) {
writer.println("\n");
}
}
writer.close();
reader.close();
os.close();
conn.connect();
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
// It's failing when accessing the above method
}
I tried running the above code and I get the following error:
java.lang.IllegalStateException: state: 2
at com.android.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:234)
at com.android.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:104)
at com.android.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:1156)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:976)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:509)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:438)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:567)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java)
It turns out that the HttpEntity.writeTo method is putting the necessary new lines but when I was printing the output to System.out, Android Studio's Logcat does not show plain new lines. I confirmed this by opening the file.tmp I was creating above and it had the proper new lines in there. It looks like there's some other error with the request since the body is valid for the server.
EDIT: Found the error in my request. I wasn't setting the Content-Type (I think I erased it while deleting some other code). I ended up using this to set the content type.
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());
I just want to send a text file and a JPEG file over the network. fortunately, i have access to both the server code and the client code. Here's my (google app engine) code.
private void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename filename = new GcsFilename("my-bucket", "my-file");
Builder fileOptionsBuilder = new GcsFileOptions.Builder();
fileOptionsBuilder.mimeType("text/html");
GcsFileOptions fileOptions = fileOptionsBuilder.build();
GcsOutputChannel outputChannel = gcsService.createOrReplace(filename, fileOptions);
byte[] buffer = new byte[1024];
InputStream reader = req.getInputStream();
BufferedOutputStream outStream = new BufferedOutputStream(Channels.newOutputStream(outputChannel));
while(true) {
int bytesRead = reader.read(buffer);
if (bytesRead == -1) {
break; // have a break up with the loop.
} else if (bytesRead < 1024) {
byte[] temp = Arrays.copyOf(buffer, bytesRead);
outStream.write(temp);
} else {
outStream.write(buffer);
}
}
outStream.close();
outputChannel.close();
}
As you can see, i use a raw InputStream to get all the data that is sent over the net.
and on the client side, i send a text file over like so: (in Android)
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpost = new HttpPost("http://my-hosted-url/postit");
MultipartEntity entity = new entity.addPart("myImageFile", new FileBody(someLogFile));
httpost.setEntity(entity);
HttpResponse response;
response = httpClient.execute(httpost);
This works just fine... sort of. the problem is that when i try to view the file/data that is sent, it has a header on top of it, as such:
--NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myString" Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit STRING_VALUE ---NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myImageFile"; filename="something.txt" Content-Type: application/octet-stream Content-Transfer-Encoding: binary
[Thu Aug 14 17:14:26 PDT 2014] then the real log starts here...
How do i get rid of the headers that is somehow stuck to the body?
What you have here is a multipart request. It is a single request where the body consists of the various parts separated by a separator string.
In your case, it's more easily viewed as:
--NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myString"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
STRING_VALUE
---NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myImageFile"; filename="something.txt"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
[binary here]
It has two parts where each part has its corresponding headers and body. I'm guessing you're interested in the bodies. You'll need to extract them.
You can either read the HTTP specification and/or the specification about multipart requests and write your own parser, or you can use some built-in (I don't know if GAE is Servlet 3.0 ready or not) or 3rd party methods. See these
How can my Servlet receive parameters from a multipart/form-data form?
Convenient way to parse incoming multipart/form-data parameters in a Servlet
How can I handle multipart/form-data POST requests in my java servlet?
i just want to upload a file using jersey rest service and Jquery ajax as client here is my code
1. HTML
<form action="rest/file/upload" method="post" enctype="multipart/form-data">
<p>
Select a file : <input type="file" name="file" />
</p>
<input type="submit" value="Upload It" />
</form>
2.Rest Service
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(#FormDataParam("file") InputStream stream) {
String uploadedFileLocation = "E:\\\\uploaded\\test.jpg";
//Session s = Session.getDefaultInstance(new Properties());
//InputStream is = new ByteArrayInputStream(<< String to parse >>);
//MimeMessage message = new MimeMessage(s, stream);
//Multipart multipart = (Multipart) message.getContent();
// save it
writeToFile(stream, uploadedFileLocation);
String output = "File uploaded to : " + uploadedFileLocation;
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
return Response.status(200).entity(output).build();
}
// save uploaded file to new location
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
byte[] image = IOUtils.toByteArray(uploadedInputStream);
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
IOUtils.write(image, out);
/*int read = 0;
byte[] bytes = new byte[1024];
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}*/
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
its working but stream includes this line also
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg"
Content-Type: image/pjpeg
how to remove this from inputstream?
need expertise answers
This string that you see is a kind of identifier added by the server to mark the start and end of the data uploaded in a form. If you dump the whole data in a text file, it will show something like this in your text file.
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: multipart/form-data; boundary=-----------------------------7dd3b827a0ddc
Content-Length: 29278
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="txt1"
Some Sample Text
-----------------------------7dd3b827a0ddc
Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg" Content-Type: image/jpeg
(Binary data not shown)
-----------------------------7dd3b827a0ddc--
The value of the boundary i.e. -----------------------------7dd3b827a0ddc is a marker that multipart form data uses to identify the start and end of data for all fields in your overall upload.
I created this sample file for you assuming one file upload and a input text named txt1.
On the data file you can see the "boundary" in header and then the boundary being used to separate the two fields in the form data. Notice the extra "--" on the last boundary. That marks the end of the file.
You need to manually parse the data and extract all of the fields. The data between the markers where you have filename="Jellyfish.jpg" is the actual binary data uploaded for your image. When you extract that data (excluding the "Content-Disposition: form-data; name="file"; filename="Jellyfish.jpg" Content-Type: image/jpeg") from between the two markers and save that data as "Jellyfish.jpg"; this will be your image.
I'm trying to upload files and posting some variables via URLConnection and multipart/form-data. But the request is empty at the server.
I've written a very basic PHP script for prototyping the solution. At the moment the code looks like that:
$uploaddir = './uploads/';
$uploadfile = $uploaddir . basename($_FILES['file1']['name']);
if(move_uploaded_file($_FILES['file1']['tmp_name'], $uploadfile)) {
echo 'success ' . $_POST['input1'] . ' ' . $_POST['input2'];
} else {
echo 'error ';
var_dump($_FILES);
echo ' ';
var_dump($_POST);
}
And my Java test code looks like:
URL url = new URL(DEST_URL);
String boundary = "---------------------" + Long.toString(System.currentTimeMillis());
PrintWriter writer = null;
URLConnection con = url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
OutputStream output = con.getOutputStream();
InputStream input = new FileInputStream(new File(FILE_PATH));
writer = new PrintWriter(new OutputStreamWriter(output));
writer.println(boundary);
writer.println("Content-Disposition: form-data; name=\"input1\"");
writer.println();
writer.println("1234");
writer.flush();
writer.println(boundary);
writer.println("Content-Disposition: form-data; name=\"input1\"");
writer.println();
writer.println("asdf");
writer.flush();
writer.println(boundary);
writer.println("Content-Disposition: form-data; name=\"file1\"; filename=\"clicknpoint.png\"");
writer.println("Content-Type: image/png");
writer.flush();
int length = 0;
byte[] buffer = new byte[1024];
for(length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
input.close();
writer.println();
writer.println(boundary);
writer.flush();
input = con.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String cur = null;
StringBuffer buf = new StringBuffer();
while((cur = reader.readLine()) != null) {
buf.append(cur);
}
System.out.println(buf.toString());
Assert.assertTrue(buf.toString().startsWith("success"));
The test case fails and prints
error array(0) {} array(0) {}
I've inspected the request with wireshark. That's what is sended:
POST /test/upload.php HTTP/1.1
Content-Type: multipart/form-data; boundary=---------------------1350394409130
User-Agent: Java/1.6.0_33
Host: localhost
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Length: 94528
---------------------1350394409130
Content-Disposition: form-data; name="input1"
1234
---------------------1350394409130
Content-Disposition: form-data; name="input1"
asdf
---------------------1350394409130
Content-Disposition: form-data; name="file1"; filename="clicknpoint.png"
Content-Type: image/png
.PNG
.
...
IHDR..............d<.....sRGB.........gAMA......a.....pHYs..........o.d....IDATx^..}.FWy......c,UU.....|.($. .B1..4%..(MC. ...(..j..."<.I.dD.&.DD.&q..Y....P..\..v.!..k...1v....m. // and so on
Any idea what's wrong?
THe problem is the invalid boundary. The boundary has to start with extra "--" and the request has to end with "--" + boundary + "--" in it's own line.
That's missing here.