Develop Reference

How do I check for duplicate file from URL before downloading

I have thousands of images in my folder on my computer and I am trying to find out how can I check if the file from given URL is already downloaded. Is is possible somehow?
This only give me size of the file.
URL url = new URL("http://test.com/test.jpg");
url.openConnection().getContentLength();
For duplicate file I use
FileUtils.contentEquals(file1, file2)
Thank you for your answers!

If you have a base URL and store files with the same filenames. You can ask the server if it's worth downloading the image again thanks to the file modification time and the If-Modified-Since HTTP Header.
File f = new File();// the file to download
HttpURLConnection con = (HttpURLConnection) new URL("http://www.test.com/"+f.getName()).openConnection();
// Add the IfModifiedSince HEADER
con.setIfModifiedSince(f.lastModified());
con.setRequestMethod("GET");
con.connect();
if(con.getResponseCode() == 304) {
System.out.println(f+ " : already downloaded");
} else {
// Download the content again and store the image again
}
It will work if the modification time of the local file has been left intact since the first download and if the server supports IfModifiedSince header.
If you don't know how to match the filename and the URL then there is no obvious way to it.
You could do some experiments with a fast HEAD request and extract some relevant informations like :
Content-Length
Last-Modified
ETag
Content-Length + Last-Modified could be a good match.
For ETags if you know how the http server builds the ETag you could try to build it on your side (on all your local files) and use it as a value to compare.
Some info on ETags:
http://bitworking.org/news/150/REST-Tip-Deep-etags-give-you-more-benefits
https://serverfault.com/questions/120538/etag-configuration-with-multiple-apache-servers-or-cdn-how-does-google-do-etag
Unfortunately ETag can be constructed with informations only visible to server (inode number) so it will be impossible for you to rebuild it.
It will certainly be easier/faster to download your files again.

If you don't download the file you can't compare it with another.
Otherwise you can store the content you downloaded in a temp file:
File temp = new File(FileUtils.getTempDirectory(), "temp");
FileUtils.copyURLToFile(url, temp);
then loop through your existing files and call:
FileUtils.contentEquals(temp, existingFile)
In the end you would want either to keep or delete the temp file.
Of course this is not very fast. If you have thousands of files, you could save their hashes in a file and use that instead of FileUtils.contentEquals.

How to make a jsp page download 2 files

I have a download.jsp page that when loaded, causes a file to download
using the following code:
String contentDisp = "attachment; filename=file_" + "."+DeptNumber+ ".txt";
response.setContentType("text/plain");
response.setHeader("Content-Disposition", contentDisp);
After this, i do some out.write(....) statements and then in the end out.flush()
which is when the user receives a download file request from the browser.
After that i use:
response.sendRedirect("landingpage.jsp");
To move the user on to the next page.
Now, I want to do the same but instead of one file, i want the page to cause 2 files to be generated for download. I did 2 changes:
1) I changed the beginning of the jsp to check for an attribute, and according to the attribute sent, generate the correct file for download.
String downloadDeptNumber = request.getAttribute("dept")==null ? "1" : request.getAttribute("dept").toString();
and
2) In the end of the page instead of redirecting to the next page, i check if the attribute was 1, i change it to 2 and redirect to the same page. If it wasn't 1, i redirect to the end page (landing page) that i used originally with only 1 download above.
if (downloadDeptNumber.equals("1"))
{
redirectUrl="download.jsp" ;
session.setAttribute("dept", "2");
}
else
{
redirectUrl= "landingpage.jsp";
session.removeAttribute("dept");
}
response.sendRedirect(redirectUrl);
But it seems i can't redirect to the same page i am on currently.
Does anyone have a solution for me ?
Thanks.

option # 1 : separate link
Give 2 separate link to download each file
option #2 : zip file
make a single zip file on server using java API and download
option #3 : Use java script
make a java script function that will open the new tab and download the each file simultaneously
Below code should run to download each file
window.open(
'http://download_file_link',
'_blank' // <- This is what makes it open in a new window.
);

Glassfish - uploading images - doing it right

I am on latest glassfish (3.1.2) - so no need for apache FileItem and no bugs with getPart(). I read that the best practice on uploading images is saving them on the file system (see here for instance). I am editing already existing code - smelly at that - so I had the idea to do :
Part p1 = request.getPart("file");
System.out.println("!!!!!P1 : " + p1);
Prints :
!!!!!P1 : File name=DSC03660.JPG,
StoreLocation=C:\_\glassfish3\glassfish\domains\domain1\generated\jsp\elkethe\upload_7cb06306_138b413999a__7ffa_00000000.tmp,
size=2589152bytes, isFormField=false, FieldName=file
newlines mine. In the code people are doing :
if (request.getParameter("crop") != null) {
// get path on the server
String outputpath = this.getServletContext().getRealPath(
"images/temp/" + session.getId() + ".jpg");
// store photo
InputStream is = p1.getInputStream();
createPhoto(is, outputpath);
session.setAttribute("photo_path", "images/temp/" + session.getId()
+ ".jpg");
response.sendRedirect("cropping");
return;
}
Where
private void createPhoto(InputStream is, String outputpath) {
FileOutputStream os = null;
try {
os = new FileOutputStream(outputpath);
// write bytes taken from uploaded file to target file
int ch = is.read();
while (ch != -1) {
os.write(ch);
ch = is.read();
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
Helpers.close(os);
}
}
Now what happens is that the file is uploaded in the StoreLocation (???) on submitting the form so apparently all this p1.getInputStream() is for naught.
My questions are :
what is StoreLocation ? How tmp are those glassfish uploads ? Where are all those parameters set ? I did read BalusC' tutorial - but there is no mention of StoreLocation (google is not very helpful either).
What would be a more professional way of handling the situation - including keeping the photos outside the webroot - but using facilities glassfish provides (if it does provide) ?
Even p1 printing so nice escapes me (it does not seem to Override toString())
Interested in tips even in how should one rename the photos etc (is this sessionID thing Right ? - check also the time trick) :
if (request.getParameter("save") != null) {
long time = System.currentTimeMillis();
String path = "images/upload/" + session.getId() + time + ".jpg";
String outputpath = this.getServletContext().getRealPath(path);
// store photo
InputStream is = p1.getInputStream();
createPhoto(is, outputpath);
// etc
}

Good practice is to pick a path on the filesystem where photos will be uploaded. Often this path is programmed to be configurable via java system property (eg: by passing -Dcom.mycompany.uploadPath=/path/to/photos/dir system property on JVM arguments).
You can also use java system propeties to find environment specific path: user.dir, user.home etc. See System Properties on Java SE Tutorial. Or to use glassfish-relative path, see glassfish system properties.
Once you have reference to Part, it's just about doing file IO to copy the uploaded file into this upload path, eg:
Part part = // obtain part somehow..
String photoFileName = // build a file name somehow..
InputStream photoInputStream = part.getInputStream();
FileOutputStream photoOutputStream = new FileOutputStream(System.getProperty("com.mycompany.uploadPath") + File.separator + photoFileName);
IOUtils.copy(photoInputStream, photoOutputStream);
// close streams here...
Code above uses apache IOUtils for convenience but feel free to write your own copy method. You should also add exception handling method

What is StoreLocation ? How tmp are those glassfish uploads ? Where are all those parameters set ?
StoreLocation is just the the java.io.File object for the FileItem's
data's temporary location on the disk. Resides in javax.servlet.context.tempdir which defaults to %GLASSFISH_HOME%\domains\domain1\generated\jsp\webApp. Those uploads are as tmp as anything (The lifetime of the file is tied to the lifetime of the FileItem instance; the file will be deleted when the instance is garbage collected - from here). Haven't yet managed to change the value of javax.servlet.context.tempdir programmatically (comment please) - it is the tempdir property of the sun-web-app element of the sun-web.xml.
What would be a more professional way of handling the situation - including keeping the photos outside the webroot - but using facilities glassfish provides (if it does provide) ?
Well a more professional way is to Use Part.write() to move the file to the desired location. Due to glassfish implementation though you can't supply an absolute path to write - a chore. I asked here.
As to where to save the file : https://stackoverflow.com/a/18664715/281545
That is for saving the file - to serve it from a location outside the app you need to define "alternatedocroot" properties in the sun-web.xml (or glassfish-web.xml).
Even p1 printing so nice escapes me (it does not seem to Override toString())
Oh yes it does
Interested in tips even in how should one rename the photos etc (is this sessionID thing Right ? - check also the time trick)
No it is not - I tend towards File#createTempFile() - anyway this is a different question asked here

Open mails in outlook from java using the protocol "mapi://"

I developp a Java application using Windows Desktop Search from which I can retrieve some information about files on my computer such as urls (System.ItemUrl). An example of such url is
file://c:/users/ausername/documents/aninterestingfile.txt
for "normal" files. This field give also urls of mail items indexed from Outlook or Thunderbird. Thunderbird's items (only available using vista and seven) are also files (.wdseml). But outlook's items urls start with "mapi://" like :
mapi://{S-1-5-21-1626573300-1364474481-487586288-1001}/toto#mycompany.com($b423dcd5)/0/Inbox/가가가가곕갘객겒갨겑곓걌게겻겨곹곒갓곅갩갤가갠가
The problem I have is opening the real item from Java in Outlook using this url. If I copy/paste it in the run dialog of Windows, it works ; it also works if I use "start" followed by the copied/pasted url in command line.
The url seems to be encoded in UTF-16. I want to be able to write such code :
String url = "mapi://{S-1-5-21-1626573300-1364474481-487586288-1001}/toto#mycompany.com($b423dcd5)/0/Inbox/가가가가곕갘객겒갨겑곓걌게겻겨곹곒갓곅갩갤가갠가";
Runtime.getRuntime().exec("cmd.exe /C start " + url);
I doesn't work and I've tried other solutions like :
String start = "start";
String url = "mapi://{S-1-5-21-1626573300-1364474481-487586288-1001}/toto#mycompany.com($b423dcd5)/0/Inbox/가가가가곕갘객겒갨겑곓걌게겻겨곹곒갓곅갩갤가갠가";
FileOutputStream fos = new FileOutputStream(new File("test.bat");
fos.write(start.getBytes("UTF16");
fos.write(url.getBytes("UTF16"));
fos.close();
Runtime.getRuntime().exec("cmd.exe /C test.bat");
without any success. Using the solution above, the file "test.bat" contains the correct url and the "start" command, but the run of "test.bat" results in the well known error message :
'■' is not recognized as an internal or external command, operable program or batch file.
Has anybody an idea to be able to open "mapi://" items from Java ?

Well, my question was a little bit tricky. But I finally found an answer and will share it here.
What I suspected was true : Windows uses UTF-16 (little endian) urls. It makes no differences working in UTF-8 when we only use paths to files such as images, text, etc. But to be able to access Outlook items, we must use UTF-16LE. If I were coding in C#, there wouldn't have been any problem. But in Java, you have to be more inventive.
From Windows Desktop Search, I retrieve this:
mapi://{S-1-5-21-1626573300-1364474481-487586288-1001}/toto#mycompany.com($b423dcd5)/0/Inbox/가가가가곕갘객겒갨겑곓걌게겻겨곹곒갓곅갩갤가갠가
And what I did is creating a temporary VB script and run it like this:
/**
* Opens a set of items using the given set of paths.
*/
public static void openItems(List<String> urls) {
try {
// Create VB script
String script =
"Sub Run(ByVal sFile)\n" +
"Dim shell\n" +
"Set shell = CreateObject(\"WScript.Shell\")\n" +
"shell.Run Chr(34) & sFile & Chr(34), 1, False\n" +
"Set shell = Nothing\n" +
"End Sub\n";
File file = new File("openitems.vbs");
// Format all urls before writing and add a line for each given url
String urlsString = "";
for (String url : urls) {
if (url.startsWith("file:")) {
url = url.substring(5);
}
urlsString += "Run \"" + url + "\"\n";
}
// Write UTF-16LE bytes in openitems.vbs
FileOutputStream fos = new FileOutputStream(file);
fos.write(script.getBytes("UTF-16LE"));
fos.write(urlsString.getBytes("UTF-16LE"));
fos.close();
// Run vbs file
Runtime.getRuntime().exec("cmd.exe /C openitems.vbs");
} catch(Exception e){}
}

File not found when browsing

I have one problem that is to upload file. It is working perfectly on my computer but fails when deploying to a server.
The system is to browse the file, then the system will zip it before uploading it to the server. When a client browse a file, the server will generate an error that the file is not found. Here is my code:
try {
//This is a code to read a zipfile.
String dir = request.getParameter("dirs");
System.out.println(dir);
String tmp = dir.replace( '\\', '/' );
System.out.println(tmp);
String inFilename = tmp;
// String inFilename = dir;
String outFilename = "c:/sms.zip";
//String outFilename = "/webapps/ROOT/sms.zip";
FileInputStream in = new FileInputStream( inFilename);
ZipOutputStream out = new ZipOutputStream(
new FileOutputStream(outFilename));
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(inFilename));
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
//End of zipping file.
//Start uploading.
SimpleFTP ftp = new SimpleFTP();
// Connect to an FTP server on port 21.
ftp.connect("xxxxx", 21, "xxx", "xxxx");
// Set binary mode.
ftp.bin();
// Change to a new working directory on the FTP server.
ftp.cwd("web");
// Upload some files.
ftp.stor(new File("sms.zip"));
ftp.disconnect();
//finish uploading
out.closeEntry();
out.close();
in.close();
response.sendRedirect("../BakMeClient/success.jsp");
}
catch (IOException e) {
System.out.println(e);
}
String dir is the location of file.
The error message is:
java.io.FileNotFoundException: D:\RELIVA\listmenu.java (The system cannot find the file specified)
Thanks for all your comments. From my observation the problem is this script is run on the server not on the client.
What I mean is let's say you browse the file for example at c:/test.txt. When you click the upload button, the form will send the path to the server and the server will find the path in its own directory and of course it will not find it.
I hope you get the idea what happened. So now: how to made it read the path at the client?

Here is definitely a problem:
// Upload some files.
ftp.stor(new File("sms.zip"));
The archive has been created at c:/sms.zip but you try to read it from the relative file location sms.zip (which is equal to ${JAVA_HOME}/sms.zip if I remember correctly The correct part is in Joachim's comment, thanks!!).
Replace these lines with
// Upload some files.
ftp.stor(new File("c:/sms.zip"));
If this doesn't help, then in addition try closing the ZipOutputStream before you send the file with FTP. There's a chance that the ZIP file has not been created yet on the file system just because the stream is still open.

There's a major misunderstanding here. You're sending local disk file system paths around instead of the actual file contents. Imagine that I am the client and I have a file at c:/passwords.txt and I give the path to you. How would you as being a server ever get its contents?
With new FileInputStream("c:/passwords.txt")? No, that is fortunately not going to happen. It will only work when both the client and server runs at physically same machine, as you have found out.
Uploading files with HTML (regardless of if it's inside a JSP file) is supposed to be done with an <input type="file"> field as follows:
<form action="upload" method="post" enctype="multipart/form-data">
<input type="file" name="file">
<input type="submit">
</form>
This way the file will be sent in the request body. As the standard Servlet API versions up to with 2.5 doesn't support mulipart/form-data requests, you need to parse the request yourself. The best way is to use Apache Commons FileUpload for this. Follow the link and read both the User Guide and Frequently Asked Questions for code examples and tips&tricks. When you're already on Servlet 3.0, then you can just use the Servlet API provided HttpServletRequest#getParts() for this. You can find here an article with code examples about that.
If you actually want to upload a complete folder with files to the server side and you don't want to use multiple <input type="file"> fields for this, then you'll need Applet or SWF for this, because this isn't possible with plain vanilla HTML. In the server side you can parse the request just the same way.

I think, if it is working in your system and not in server, there must be problem with server settings.
Or you can check following things
Need to check path you are working on.
Before uploading, try to list the files in that directory, once you generate ZIP file.
Check for permissions.

Your outFilename must be found in the web. Like: "http://www.sample.com/sms.zip" or the likes..