I just want to send a text file and a JPEG file over the network. fortunately, i have access to both the server code and the client code. Here's my (google app engine) code.
private void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename filename = new GcsFilename("my-bucket", "my-file");
Builder fileOptionsBuilder = new GcsFileOptions.Builder();
fileOptionsBuilder.mimeType("text/html");
GcsFileOptions fileOptions = fileOptionsBuilder.build();
GcsOutputChannel outputChannel = gcsService.createOrReplace(filename, fileOptions);
byte[] buffer = new byte[1024];
InputStream reader = req.getInputStream();
BufferedOutputStream outStream = new BufferedOutputStream(Channels.newOutputStream(outputChannel));
while(true) {
int bytesRead = reader.read(buffer);
if (bytesRead == -1) {
break; // have a break up with the loop.
} else if (bytesRead < 1024) {
byte[] temp = Arrays.copyOf(buffer, bytesRead);
outStream.write(temp);
} else {
outStream.write(buffer);
}
}
outStream.close();
outputChannel.close();
}
As you can see, i use a raw InputStream to get all the data that is sent over the net.
and on the client side, i send a text file over like so: (in Android)
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpost = new HttpPost("http://my-hosted-url/postit");
MultipartEntity entity = new entity.addPart("myImageFile", new FileBody(someLogFile));
httpost.setEntity(entity);
HttpResponse response;
response = httpClient.execute(httpost);
This works just fine... sort of. the problem is that when i try to view the file/data that is sent, it has a header on top of it, as such:
--NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myString" Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit STRING_VALUE ---NNqarc4FsG0G8hUzd82A6TCjgzKH Content-Disposition: form-data; name="myImageFile"; filename="something.txt" Content-Type: application/octet-stream Content-Transfer-Encoding: binary
[Thu Aug 14 17:14:26 PDT 2014] then the real log starts here...
How do i get rid of the headers that is somehow stuck to the body?
What you have here is a multipart request. It is a single request where the body consists of the various parts separated by a separator string.
In your case, it's more easily viewed as:
--NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myString"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
STRING_VALUE
---NNqarc4FsG0G8hUzd82A6TCjgzKH
Content-Disposition: form-data; name="myImageFile"; filename="something.txt"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
[binary here]
It has two parts where each part has its corresponding headers and body. I'm guessing you're interested in the bodies. You'll need to extract them.
You can either read the HTTP specification and/or the specification about multipart requests and write your own parser, or you can use some built-in (I don't know if GAE is Servlet 3.0 ready or not) or 3rd party methods. See these
How can my Servlet receive parameters from a multipart/form-data form?
Convenient way to parse incoming multipart/form-data parameters in a Servlet
How can I handle multipart/form-data POST requests in my java servlet?
Related
A webservice returns a MIME file as InputStream with following content. I use Java Apache HTTPClient in order to make a request:
MIME-Version:1.0
Content-Type:multipart/mixed;
boundary="----=_Part_58_1750763977.1605815692305"
------=_Part_58_1750763977.1605815692305
Content-Type: application/octet-stream; name=preview.pdf
Content-ID: response-1
Content-Disposition: attachment; filename=preview.pdf
%PDF-1.7
[...]
------=_Part_67_626667127.1605818243111
Content-Type: application/octet-stream; name=thumbnail.jpg
Content-ID: response-2
Content-Disposition: attachment; filename=thumbnail.jpg
------=_Part_58_1750763977.1605815692305
Content-Type: application/octet-stream; name=report.xml
Content-ID: response-3
Content-Disposition: attachment; filename=report.xml
How can I now convert these chunks to single files? I tried javax.Mail and MIME file, but didn't succeed.
I managed to parse the message as mime and save the attachments as file:
HttpResponse response = client.execute(post);
Session s = Session.getInstance(new Properties());
MimeMessage myMessage = new MimeMessage(s, response.getEntity().getContent());
Multipart multipart = (Multipart) myMessage.getContent();
for (int i = 0; i < multipart.getCount(); i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
InputStream is = bodyPart.getInputStream();
File f = new File("tmp/" + bodyPart.getFileName());
FileOutputStream fos = new FileOutputStream(f);
byte[] buf = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buf))!=-1) {
fos.write(buf, 0, bytesRead);
}
fos.close();
}
by using
compile([group: 'tech.blueglacier', name: 'email-mime-parser', version: '1.0.5'])
I'm trying to configure and send a multipart request like the following one:
------boundary
Content-Disposition: form-data; name="before"; filename="blob"
Content-Type: application/vnd...+json;type=some_type
{some JSON}
------boundary
Content-Disposition: form-data; name="after"; filename="blob"
Content-Type: application/vnd...+json;type=some_type
{another JSON}
------boundary--
So I tried to configure a request, as in the code below
RestAssuredConfig config = RestAssured.config().multiPartConfig(
new MultiPartConfig().defaultCharset(StandardCharsets.UTF_8).
defaultBoundary("--boundary--"));
MultiPartSpecification m1 new MultiPartSpecBuilder(
new ObjectMapper().writeValueAsString(some_JSON_transformed_to_HashMap)).
fileName("blob").controlName("before").
mimeType(ContentType.TEXT.getAcceptHeader()).
header("ContentType", "application/vnd...+json;type=some_type").build();
MultiPartSpecification m2 = new MultiPartSpecBuilder(
new ObjectMapper().writeValueAsString(another_JSON_transformed_to_HashMap)).
fileName("blob").controlName("after").
mimeType(ContentType.TEXT.getAcceptHeader()).
header("ContentType", "application/vnd...+json;type=some_type").build();
RequestSpecification request = RestAssured.given().multiPart(m1).multiPart(m2).
config(config).
.header("Content-Type", "multipart/form-data; boundary=" + config.getMultiPartConfig().defaultBoundary());
request.post("some_url");
But when I try to execute it, server says that resource is invalid, but I believe, that JSONs is correct, so I suppose my multipart configuration is incorrect. How should I configure a request?
You could try something similar to below code .
given().auth().preemptive()
.basic("Jirausername", "Jirapassword")
.header("X-Atlassian-Token", "nocheck")
.multiPart(new File("/home/users/cat.log"))
.when().post("http://localhost:8181/rest/api/2/issue/STS-223/attachments");
I have a java server using sockets. I have an html file which contains a form that can be used to upload a file and send it. How can the server download that file without blocking. The browser never closes the connection and it just hangs in there I have beeping using readLine() from the input stream but it blocks. is there any way around this?
I appreciate any help here.
Thanks
please clarify your requirement, it seems total chaos in your steps.
did your program run as client side or server side?
could you show your code? and show what blocked you?
It is much better if you can draw a working-flow of your program .
Thanks for your supplement, do you want to upload a files through HTTP(writing in socket way). For this requirement, you could check the link link for how HTTP works
[RFC 1867] (https://www.rfc-editor.org/rfc/rfc1867)
Below is the package of how socket way do:
Suppose the server supplies the following HTML:
<FORM ACTION="http://server.dom/cgi/handle"
ENCTYPE="multipart/form-data"
METHOD=POST>
What is your name? <INPUT TYPE=TEXT NAME=submitter>
What files are you sending? <INPUT TYPE=FILE NAME=pics>
</FORM>
and the user types "Joe Blow" in the name field, and selects a text
file "file1.txt" for the answer to 'What files are you sending?'
The client might send back the following data:
Content-type: multipart/form-data, boundary=AaB03x
--AaB03x
content-disposition: form-data; name="field1"
Joe Blow
--AaB03x
content-disposition: form-data; name="pics"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
If the user also indicated an image file "file2.gif" for the answer
to 'What files are you sending?', the client might client might send
back the following data:
Content-type: multipart/form-data, boundary=AaB03x
--AaB03x
content-disposition: form-data; name="field1"
Joe Blow
--AaB03x
content-disposition: form-data; name="pics"
Content-type: multipart/mixed, boundary=BbC04y
--BbC04y
Content-disposition: attachment; filename="file1.txt"
Nebel & Masinter Experimental [Page 9]
RFC 1867 Form-based File Upload in HTML November 1995
Content-Type: text/plain
... contents of file1.txt ...
--BbC04y
Content-disposition: attachment; filename="file2.gif"
Content-type: image/gif
Content-Transfer-Encoding: binary
...contents of file2.gif...
--BbC04y--
--AaB03x--
I have server running on port. I want it to gather the data sent by a POST request which carries a file (Not just text but png image).
for example here is a server:
public void run() {
try {
InputStream is = insocket.getInputStream();
PrintWriter out = new PrintWriter(insocket.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String line;
line = in.readLine();
String request_method = line;
System.out.println("HTTP-HEADER: " + line);
line = "";
// looks for post data
int postDataI = -1;
while ((line = in.readLine()) != null && (line.length() != 0)) {
System.out.println("HTTP-HEADER: " + line);
if (line.indexOf("Content-Length:") > -1) {
postDataI = new Integer(
line.substring(
line.indexOf("Content-Length:") + 16,
line.length())).intValue();
}
}
String postData = "";
// read the post data
if (postDataI > 0) {
char[] charArray = new char[postDataI];
in.read(charArray, 0, postDataI);
postData = new String(charArray);
}
out.println("HTTP/1.0 200 OK");
out.println("Content-Type: text/html; charset=utf-8");
out.println("Server: MINISERVER");
// this blank line signals the end of the headers
out.println("");
// Send the HTML page
out.println("<H1>Welcome to the Mini Server</H1>");
out.println("<H2>Request Method->" + request_method + "</H2>");
out.println("<H2>Post->" + postData + "</H2>");
out.println("<form name=\"input\" action=\"form_submited\" method=\"post\">");
out.println("Username: <input type=\"text\" name=\"user\"><input type=\"submit\" value=\"Submit\"></form>");
out.close();
insocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
How can it be changed to safe a file (Uploaded through HTTP/1.1 POST REQUEST) form) to disk using this server.
Thank you for your time
I am using org.apache.http.HttpEntity for doing a multipart/form data POST to HTTPURLConnection to upload a file.
Here is the code that I am using.
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
String part1 = "\n{\"name\":\"test.txt\",\"creationTime\":1527023510389,\"fileUri\":\"/storage/test.txt\"}";
File file = new File("/storage/test.txt");
HttpEntity entity = MultipartEntityBuilder.create()
.setMode(HttpMultipartMode.BROWSER_COMPATIBLE)
.addBinaryBody("data", part1.getBytes(), ContentType.APPLICATION_JSON, "data.txt")
.addBinaryBody("file", file, ContentType.TEXT_PLAIN, filename)
.setBoundary(boundaryString)
.build();
OutputStream os = conn.getOutputStream();
entity.writeTo(os);
I see that the body is being posted as the following.
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
The problem is that the server requires a new line between the Content-Type and the contents of the first part. I've tried adding extra "\n" to the beginning contents (as seen but it gets erased when using HttpEntity.writeto().
The output that I want is the following:
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
I attempted to modify rewriting the output but not sure if this is the best way to do it by storing in a temporary file. The files I will be working with will be up to 20mb if that makes any difference.
entity.writeTo(new FileOutputStream("file.tmp"));
BufferedReader reader = new BufferedReader(new FileReader("file.tmp"));
OutputStream os = conn.getOutputStream();
PrintWriter writer = new PrintWriter(new BufferedOutputStream(os));
String str;
while ((str = reader.readLine()) != null) {
writer.println(str);
if (str.contains("Content-Type: ")) {
writer.println("\n");
}
}
writer.close();
reader.close();
os.close();
conn.connect();
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
// It's failing when accessing the above method
}
I tried running the above code and I get the following error:
java.lang.IllegalStateException: state: 2
at com.android.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:234)
at com.android.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:104)
at com.android.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:1156)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:976)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:509)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:438)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:567)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java)
It turns out that the HttpEntity.writeTo method is putting the necessary new lines but when I was printing the output to System.out, Android Studio's Logcat does not show plain new lines. I confirmed this by opening the file.tmp I was creating above and it had the proper new lines in there. It looks like there's some other error with the request since the body is valid for the server.
EDIT: Found the error in my request. I wasn't setting the Content-Type (I think I erased it while deleting some other code). I ended up using this to set the content type.
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());
Hello fellow java developers. I receive a response with headers and body as below, but when I try to decompress it using the code below, it fails with this exception:
java.io.IOException: Not in GZIP format
Response:
HTTP/1.1 200 OK
Content-Type: text/xml; charset=utf-8
Content-Encoding: gzip
Server: Jetty(6.1.x)
▼ ═UMs¢0►=7┐ép?╙6-C╚$╢gΩ↓╟±╪₧∟zS╨╓╓♦$FÆ╒÷▀G┬╚╞8N≤╤Cf°►╦█╖╗o↨æJÄ+`:↓2
♣»└√S▬L&?∙┬_)U╔|♣%ûíyk_à\,æ] hⁿ?▀xΓ∟o╜4♫ù\#MAHG?┤(Q¶╞⌡▌Ç?▼ô[7Fí¼↔φ☻I%╓╣Z♂?¿↨F;x|♦o/A╬♣╘≡∞─≤╝╘U∙♥0☺æ?|J%à{(éUmHµ %σl┴▼Ç9♣┌Ç?♫╡5╠yë~├╜♦íi♫╥╧
╬û?▓ε?╞┼→RtGqè₧ójWë♫╩∞j05├╞┘|>┘º∙↑j╪2┐|= ÷²
eY\╛P?#5wÑqc╙τ♦▓½Θt£6q∩?┌4┼t♠↕=7æƒ╙?╟|♂;║)∩÷≈═^╛{v⌂┌∞◄>6ä╝|
Code:
byte[] b= IOUtils.toByteArray(sock.getInputStream());
ByteArrayInputStream bais = new ByteArrayInputStream(b);
GZIPInputStream gzis = new GZIPInputStream(bais);
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
System.out.println("read: "+readed);
}
Please advise.
Thanks,
Pradeep
The MIME header is NOT in the GZIP format, it's in plain text. You have to read that first before you can decompress the stream.
Also, why not just use this:
InputStream in = sock.getInputStream();
readHeader(in);
InputStream zin = new GZIPInputStream(in);
There are libraries for all of this. You can use, for example, Apache HTTP Components, or you can read its open source to see what it does. At very least, read the relevant specification.
I second bmarguiles' answer.
Only the body (response-body in the RFC) is compressed, so you only need to decompress the part that is after the \r\n\r\n.
Generally speaking, you can cut the response in half by that double CRLF, and only decompress the second half.