I have a java applet for uploading files to server.
I want to display the % of data sent but when I use ObjectOutputStream.write() it just writes to the buffer, does not wait until the data has actually been sent. How can I achieve this.
Perhaps I need to use thread synchronization or something. Any clues would be most helpful.
Don't use ObjectOutputStream. It's for writing serialized Java objects, not for writing raw binary data. It may indeed block the stream. Rather just write directly to the OutputStream of the URL connection.
That said, the code looks pretty overcomplicated. Even after re-reading several times and blinking my eyes countless times, I can't get it right. I suggest you to send those files according the multipart/form-data encoding with help of Commons HttpClient. You can find here a basic code example. You just have to modify Part[] parts to include all the files. The servlet on the other side can in turn use Commons FileUpload to parse the multipart/form-data request.
To calculate the progress, I'd suggest to pick CountingOutputStream of Commons IO. Just wrap the OutputStream with it and write to it.
Update: if you don't like to ship your applet with more 3rd party libraries (which I imagine is reasonable), then have a look at this code snippet (and the original question mentioned as 1st link) how to create a multipart/form-data body yourself using URLConnection.
What I was looking for was actually:
setFixedLengthStreamingMode(int contentLength)
This prevents any internal buffering allowing me know exactly the amount of data being sent.
Related
Good afternoon everyone,
First of all, I'll say that it's only for personal purpose in a certain way, it's made to use for little projects to improve my Java knowledge, but my idea is to make this kind of things to understand better the way developers works with sockets and bytes, as I really like to understand this kind of things better for my future ideas.
Actually I'm making a lightweight HTTP server in Java to understand the way it works, and I've been reading documentation but still have some difficulties to actually understand part of the official documentation. The main problem I'm facing is that, something I'd like to know if it's related or not, the content-length seems to have a higher length than the one I get from the BufferedReader. I don't know if the issue is about the way chars are managed and bytes are being parsed to chars on the BufferedReader, so it has less data, so probably what I have to do is treat this part as a binary, so I'd have to read the bytes of the InputStream, but here comes the real deal I'm facing.
As Readers reads a certain amount of bytes, and then it stops and uses this as buffer, this means the data from the InputStream is being used on the Reader, and it's no longer on the stream, so using read() would end up on a -1 as there aren't more bytes to read. A multipart is divided in multiple elements separated with a boundary, and a newline that delimiters the information from the content. I still have to get the information as an String to process it, but the content should be parsed into a binary data, and, without modifying the buffer length, implying I'd require knowledge about the exact length I require to get only the information, the most probably result would be the content being transferred to the BufferedReader buffer. Is it possible to do it even with the processed data from the BufferedStream, or should I find a way to get that certain content as binary without being processed?
As I said, I'm new working with sockets and services, so I don't exactly know which are the possibilities it's another kind of issue, so any help would be appreciated, thank you in advance.
Answer from Remy Lebeau, that can be found on the comments, which become useful for me:
since multipart data is both textual and binary, you are going to have to do your own buffering of the socket data so you have more control and know where the data switches back and forth. At the very least, since you can read binary data directly from a BufferedInputStream, and access its internal buffer, you can let it handle the actual buffering for you, and it is not difficult to write a custom readLine() method that can read a line of text from a BufferedInputStream without using BufferedReader
Bear with me as I know this question seems easily searchable but I've been stuck on it for some time and would greatly appreciate a thorough read-through before answering. I'm trying to write a very basic Java server, not to be scaled or implemented but just to understand networking. What is the best (in this case, simplest) method of parsing a request header to extract the filename? My current code uses a Socket to establish the connection, and in doing research I've seen:
Reading the socket's inputstream into a scanner/buffered reader and manually parsing it (by the leading / before the filename, I'm assuming)
Using default methods of HttpServletRequest (how do you generate an object of this type? Isn't an interface non-instantiable? Documentation is unclear about this)
Something with a "Content-Disposition Header"?
Thanks again for the help, these http API's seem very similar/overlapping to a rookie network programmer.
I am customizing a large COTS content management system known as Confluence.
Confluence returns many different types of httpservletresponses (text/ascii, image/png, image/jpg, microsoft powerpoint files, PDF files, etc...).
I have written a servletfilter that attempts to modify all responses sent back to the client by writing out a small set of bytes. This works well for the most part. However, I have to continuously check for special cases like powerpoint files, or PDFs, PNGs, etc.. If the user happens to be downloading such content I do not modify the response at all. Modifying the response breaks stream of powerpoint bytes or PDF bytes that are in the process of being served to the client. By simply checking for these special cases and not writing out any of my bytes my problem is solved. But I feel the bigger problem is there could be many many more cases I am not thinking of (perhaps audio and video) or who knows what. I will have to continue playing the game of checking for these special cases as I learn of them.
I was wondering if there is a smarter way to handle this.
I did a google and I ran into this example.
I'm looking for something along the lines of this example, but I was hoping someone could explain to me what's going on behind the scenes and if I can solve this problem in a smarter way.
The filter example is sort of incomplete, but the gist of what it seems to be doing is buffering the entire response in to a byte array, with which you can do whatever you want later. I think the implication is that you might extend this filter, then call getData() after the filter chain fires, and then perform processing.
You don't speak to what you're doing, or why the content type matter, or why "special" content types that you don't care about (that you just pass through) matter.
What you can do, is you could create a registry of content type handlers to classes. Then, as you detect the content type of the outbound request, you can dispatch to the appropriate handler. These handlers can be simply represented as a map of content type -> class name of the handler, with a default pass through "do nothing" handler for any content type that is not registered. You can load that map from a properties file, filter configuration, or a table in the database.
While it may seem attractive to just buffer the entire output stream and then act upon it, I would recommend against it. Imagine the memory pressure if the user is downloading a large (10's to 100's of MB) PDF or video or something else. Perhaps most of your content is appropriate to be buffered, but there may well be some that are not.
Of course your handler can implement many of the portions of the filter chain, and act as a proxy filter, so your handlers can do anything a filter can do.
Also, your filter may interfere with higher order HTTP processing (notably chunk delivery, range support, Etag and caching support, etc.). That stuff can be a pain to have to redo.
I'm reading up on non-blocking I/O as I'm using Akka and Play and blocking is a bad idea if avoidable in that context as far as I can read, but I can't get this to work together with my use case:
Get file over network (here alternatives using nio exist, but right now I'm using URL.openStream)
Decrypt file (PGP) using BouncyCastle (here I'm limited to InputStream)
Unzip file using standard Java GZIP (limited to InputStream)
Read each line in file, which is a position based flat file, and convert to a Case Classes (here I have no constraints on method for reading, right now scalax.io.Resource)
Persist using Slick/JDBC (Not sure if JDBC is blocking or not)
It's working right now basically using InputStreams all the way. However, in the interest of learning and improving my understanding, I'm investigating if I could do this with nonblocking IO.
I'd basically like to stream the file through a pipeline where I apply each step above and finally persist the data without blocking.
If code is required I can easily provide, but I'm looking of a solution on a general level: what do I do when I'm dependent on libraries using java.io?
I hope this helps with some of your points:
1/2/3/4) Akka can work well with libraries that use java.io.InputStream and java.io.OutputStream. See this page, specifically this section: http://doc.akka.io/docs/akka/snapshot/scala/io.html
A ByteStringBuilder can be wrapped in a java.io.OutputStream via the asOutputStream method. Likewise, ByteIterator can we wrapped in a java.io.InputStream via asInputStream. Using these, akka.io applications can integrate legacy code based on java.io streams.
1) You say get a file over the network. I'm guessing via HTTP? You could look into an asynchronous HTTP library. There are many fairly mature async HTTP libraries out there. I like using Spray Client in scala as it is built on top of akka, so plays well in an akka environment. It supports GZIP, but not PGP.
4) Another option: Is the file small enough to store in memory? If so you need not worry about being asynchronous as you will not be doing any IO. You will not be blocking whilst waiting for IO, you will instead be constantly using the CPU as memory is fast.
5) JDBC is blocking. You call a method with the SQL query as the argument, and the return type is a result set with the data. The method must block whilst performing the IO to be able to return this data.
There are some Java async database drivers, but all the ones I have seen seem unmaintained, so I have't used them.
Fear not. Read this section of the akka docs for how to deal with blocking libraries in an akka environment:
http://doc.akka.io/docs/akka/snapshot/general/actor-systems.html#Blocking_Needs_Careful_Management
Decrypt file (PGP) using BouncyCastle (here I'm limited to InputStream)
As you are limited to an InputStream in this step you've answered your own question. You can do the part involving the network with NIO but your step (2) requires an InputStream. You could spool the file from the network to disk using NIO and then use streams from then on, for unzipping and decrypting (CipherInputStream) ... still blocking in theory but continuous in practice.
I know this isn't non blocking IO exactly, but I think you should look at composing Futures (or Promsies) with map which is non-blocking in the Playframework sense of things.
def getFile(location: String): File = { //blocking code}
def decrypt(file: File): File = ..
def unzip(file: File): PromiseFile = ..
def store(file: File): String = ..
def result(status: String): SimpleResult[Json] = ..
AsyncResult{
Promise.pure(getFile("someloc")) map decrypt map unzip map store map result
}
I'm currently writing some update polling stuff. I try to avoid writing even a simple REST-Interface for this (we're using much REST, still I'm not sure this is necessary here. Why write an interface for functionality already there?)
My idea was to open an HttpUrlConnection and check headers for file's last modified date. Apache obviously sends "Last-Modified" date in UTC. After checking the header I'd close the connection without actually retrieving the body. I only fear that this might bring up errors in Apache log, which would be quite inconvenient. I just wanted to ask for you opinion. Do you think this might work? Better ideas?
(I need system proxy support, so my only option seems to be HttpUrlConnection.)
Regards,
Stev
If you look at the HTTP protocol, you'll see that it has a HEAD request which does just what you need. The default for HTTP requests in the Java runtime is GET and it's not really easy to change that.
Have a look at HttpClient for a framework which allows you to send any kind of request.
You are almost right but your task is even simpler that what you are explaining. There is special HTTP method named HEAD. You just have to create the same request you need to retrieve your data but use HEAD instead of GET
This sounds pretty much, what the HEAD method in HTTP is for.
Citing from Wikipedia:
HEAD
Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
Try just sending an http HEAD request. See here: http://blog.mostof.it/what-is-a-http-head-request-good-for-some-uses/
http://www.grumet.net/weblog/archives/http-head-example.html