We have an enterprise application which saves some files on a network share (\\physicalfileserver\files\). We need to use a network share because the application is installed on multiple application server instances distributed over multiple physical server connected via LAN. Sometimes, we need to create a copy of certain files on the network share itself and save it in same the directory (on the same network share) with a different name or in a new directory (Again on the same network share). What is the most efficient way to create a copy of the file? I do not want to create any IO traffic on the network as the source and destination files reside on the same server. It also increases the time taken to create a copy of the file. This however, is difficult from What I have seen so far.
Reading files in input stream and writing them back on the server through an output stream with a different path/name.
FileUtils.copyFile() from apache commons.
NIO Channels library of JAVA (java.nio.channels.FileChannel.transferFrom())
However, all of them seem to copy the file to the server executing the program and then copy the data back to network share which isn't really required.
What can the gurus suggest to be the best possible approach to handle this situation? I can only use JAVA 1.5/1.6 due to some restrictions but am open to suggestions using higher java version as well.
The application is installed on a Windows 2008 server and the network share is also present on a Windows 2008 server.
Thanks
If you want to avoid network traffic you could write a small server in Java that would allow you to copy files locally (this would run on file server itself). Securing this would be quite important as this could pose huge security risk if somebody could use it to overwrite system files.
Related
How do you decide which protocol you've to use to transfer/move/copy files across servers?
I'm writing a service to copy files from a Windows server file system to a oracle-based database.
How about the File Transfer Protocol for transferring files.
Since Oracle doesn't store files, I suspect you need more than just transferring files. I suspect you will need to use JDBC in any case.
I am not sure it's clear what you are building and why is it it a java question?
Do you need to implement a custom protocol in Java to transfer files for a web service you are building? Or can you use an off the shelf product and a bunch of shell scripts to take care of the file transfer?
If it's a custom thing you need to build in Java (the question is tagged with java) then more questions:
Is it a web app that provides end-users with GUI allowing them to submit files?
- you need to take care of file transfer using http post. Apache httpclient library can help you there.
Is it a SOAP Web Service that saves incoming data into oracle?
- again then SOAP implementations provide answer to that.
Is it a command-line tool running on Windows machine that talk to an Oracle database ?
-then protocol is irrelevant, as you can just read the files using file.io libs and use jdbc to save them into oracle.
Is it a custom server with a yet-to-be built protocol and subsequently a custom client that will use that protocol?
- then I would use binary over https for my implementation of the custom server/service.
4.If you are doing everything using COTS, FTP is simple and fast, but it's not secure and it's not reliable. SFTP will almost always going to be better.
http://geekswithblogs.net/bvamsi/archive/2006/03/23/73147.aspx
5.If your question is really about how to save files into Oracle database using jdbc, then you need to find a way to stream them (or you will run out of memory).
E.g. this may be helpful http://www.coderanch.com/t/415625/JDBC/databases/save-files-oracle-database-java
My company processes data that comes from our clients. Our clients submit data by pushing it to an sftp server we maintain. Our code (mostly in Java, Scala and Node.js with a little of Python/shell) then hits this servers in read-only mode and does bunch of data processing after downloading these files.
We are frustrated by this system because not only Java libraries aren't that great for doing sftp but even the node one does not work in Windows which some developers use so we have some hacky shell scripts to sync files from sftp.
What are some good architectural patterns to solve this problem? We want a secure internal file server that can be accessed easily by good OS agnostic libraries and that has good tools for browsing through a file-viewer?
Expanding on Brad's question, "why make you internal interface SFTP"? ...
I'd run rsync against your SFTP server to keep a working copy of the client files in your preferred environment. From there go with whatever tools make your crew most comfortable.
I have a java application which is used many computers in my office network. This is a java swing application, and when there is an update most of the time I have to update each computer. Currently there is a mechanism to update these applications automatically but it seems like it is not working properly, that is I have a simple table in the database which hold the released versions number and when the application starting it checks its' versions against the database. If it is not tally then it downloads the whole application from the FTP server which we have installed in our network.
Recently thought to create a server to do this task. But, I don't know whether it is a good practice to do.
My idea is, there is a server which can get the md5 hashes of each file in the FTP server and send that hash list and file list to its' clients. When the clients (my swing application) get this lists it compares those hashes against its' files and if there is a mismatch client can download that file from the FTP server.
Please tell me that is this a good method to do for updating a java application?
You should use Java Webstart.
It's designed for exactly this scenario: automatic downloading and update of an application from the internet and/or intranet.
It even does clever stuff like application versioning and ensuring that new files only get downloaded when they are needed through intelligent caching.
Another option:
Since this is in an internal network. If you don't want to use webstart (best option).
Create a small front end application that use a URLClass loader and always load it on the fly. Since this is an internal network unless the application is huge. It should take less then a couple of seconds to launch.
Hi I have this little big problem I have a legacy VB6 desktop application that connects to a MS Access database hosted in a local Ubuntu server machine that is being shared to the LAN as a SMB share and I have a Tomcat web application hosted in a Windows based VPS, these are
Requirements
Read information from the MS Access db and show it in the webapp (On URL visit and almost real-time accuracy).
Update the MS Access db with information received through the website.
Facts
20Mb < db size
Shared with 15 ~ 25 users.
Constant local update and querying.
The file size is small because It is being truncated as it grows tool large( > 100Mb)
Now I know that the arquitecture isn't the best and that MS Access is at its limits, a migration to full Java and MySQL is on the way, but it's going to take a long time....in the meantime I need a way to implement that feature, here are my options.
Option 1
Access the db in ubuntu server through a VPN directly from the webapp.
Cons
Is it possible?.
Slow connection.
May lock the db MORE frequently as it happens quite often locally.
Option 2
Have a local webapp run in Ubuntu server that exposes the db as a REST API, so updates would be handled by the local webapp.
Cons:
Hard to use MS Access in a UNIX environment, looking at unixODBC and FreeTDS, but so far I haven't been able to use it.
Well, writing the whole app and securing the server.
Option 3
Any suggestions?
Thank you if you read this far, any help is really appreciated.
Unless I missed something in your description, I think you might be confusing the differences between an API and a library. Basically, ODBC is an API which is implemented as a library and commonly used on Windows based machines through additional data access libraries like ADO and ADO.NET. I mention this since you referred to unixODBC as a solution. It would not be a complete solution as there is more to it than just the API alone.
In simple terms, the database file you created with Microsoft Access is a .MDB flat file database (ok, there is a little bit more to it, but in terms of treating it as a database, that is all that matters here). If you know how the structure of the .MDB file works, you could write your own library that reads/writes to it. Of course, this is not trivial and on the Windows platforms, this is provided for you by Microsoft using the libraries included in the OS. This is also referred to as a JET driver and database. JET is the database format that the .MDB file implements and is used by Access and other applications via the correspondingly named JET drivers.
So to find an equivalent option for non-Windows platforms, you need some sort of library that knows how to natively read/write to the .MDB file directly. If you are trying to use the .MDB file at the same time from an Access application, then you need to make sure the library you choose supports simultaneous mutli-user access to the database.
In a quick search, there do appear to be some solutions that I could find. The first one appears that it might have some functional limitations. The second appears to be a commercial product.
MDB Tools
Easysoft JET/Access Driver
I am searching a library or jar file for making a system monitoring program. I found that Java provides jConsole, but I need some other jar file name.
actually i am making a web service in which i have ip address and port no of remote computer and based on that i need a system process list of that computer, and that web service i will call from the google apps engine that will display the whole process list. so i need name of jar file that can i place on server and that give me the process list.
I've built such a system before and JMX (java management extensions) was very useful.
http://www.oracle.com/technetwork/java/javase/tech/javamanagement-140525.html
examples and tutorials:
http://docs.oracle.com/javase/6/docs/technotes/guides/jmx/
You can install small agents on the machines you wish to monitor. These will gather information and either push it to a main monitoring server or the monitoring server will pull data whenever it wants from those monitored clients.
You will also need to have a database to store which attributes on which machines nodes you are monitoring. You can serialize the attributes with xstream, which creates an xml out of java objects and vice versa. You can store these in the database.
The attributes can be monitored by creating beans on the monitored machines.
You might also want to give Nagios a try - a commercial solution.
There is no jar file that you can get for free and just add it to your project (at least none that I know of right now).
It's a bit hard to understand what you are looking for, given that you din't provide much information. It sounds like you might be interested in JMX.