hi
i want to perform operations on files like rename, copy and etc.
these are not local files. they are located on remote computers.
I have 2 options:
1. run some kind of a telnet client (a framework that i already have in my system) from the java code. connect to the remote computer and perform a cmd operation.
2. perform a regular java.io operation on the remote path.
the problem with 1 is that its not cross platform (only theoretical problem for me), and that i generally dont want to use this telnet framework.
the problem with 2 is that large operations on remote files is slower, compared to same operation being performed on the machine itself with telnet.
am i right?
any other options?
any additional inputs?
thanks.
If you can deploy your application to the remote computer, you can simply write your own little client and server for these file operations.
Related
I am using Cassandra DB in my java application. Am using Thrift client to connect Cassandra from my java application. If the Cassandra disk get full means it automatically terminates. So from my java program i could not find the correct error why the Cassandra is down.
So how to avoid the auto termination of Cassandra or is their any way to identify the disk full error ?
Also i dont have physical access to cassandra drive. Its running in some other remote machine.
Disk errors and, in general, generic hardware/system errors are not usually properly handled in any application. The database should only provide as much durability as possible in such scenarios and it is the correct behavior - shut down and break as little as possible.
As for your application - if you can not connect to the database, there is no difference as to what caused an error. You app will not work anyway.
There are special tools that can monitor your machine, i.e. Nagios. If you are the administrator of that server, use such applications. When the disk is getting filled up you will receive an email or text. Use such tools and don't break an open door by implementing several hundred of lines of code to handle random and very rare situations.
Setup ssh access to Casandra machine and use some ssh client like JSch to run df /casandra/drive (if Linux) or fsutil volume diskfree c:\casandra\drive (if Windows) from your Java client. Capture output that is simple and parse to obtain the free disk space. That way your application will monitor that is happening there and probably should alert the user and refuse to add data if there is an out of disk space threat.
You can also use standard monitoring tools or setup server side script to send the message if the disk space low. However this will not stop your application from crashing, you need to take actions after you see that the disk space is low.
I have mounted a NFS mount in my local machine (So, I suppose my machine is the NFS client) and a file is being written in the NFS mount by the remote machine hosting the NFS mount (NFS server). Now, how will I be able to detect using JAVA that the file being written to in the remote machine has been closed before me reading from my local machine?
Or, are there any alternative ways to achieve what I want?
Edit: I would also like to mention that I have no control over the remote NFS server. I can only mount in read only mode.
As you stated that you have no control over the NFS server nor the program updating the file on the NFS server, there is not a lot you can do. The only thing you can really make use of is the fact that when a file is updated its modification time is updated. This should be available to your java client using File.lastModified(). If your application remembers the last modification date, you can then at least know if there is data to process since you last read it. It's definitely not perfect. There are race conditions etc, but depending on what exactly your application needs/does, it may be enough. You can't tell if the file is closed or open on the server.
If the file modifications are slow to appear, you could also mount using the noac mount option. While this hurts NFS performance on a regular NFS client, it may work for you if your machine is not using a lot of other NFS files on that drive. See "man nfs".
As a side note, I can't see that file locking will help as you gave no indication that the application on the NFS server uses locks anyway.
I need to create a small utility. Using Java, remotely (from another machine) I need to search for files (whether particular files exist or not). I would know exact names of files, or if not, I would know their partial names (so I will be using wildcards).
The problem is: I cannot install anything on that machine (it's a corporate box), but I am allowed FTP, Telnet, SSH login, etc. I would not want to use FTP because there are hundreds of directories and there's no way to know ahead of time where the files are each time (using ftp command LS on each directory every time is simply unproductive). So the best would be to use internal unix 'find' command. Or a similar approach.
I need the best solution to the problem: how can I search for files (from hundreds of thousands files in many different folders) remotely on another machine.
Jsch will allow you to connect quite easily to a remote machine and execute operations using SSH. There is not a lot of documentation, but these examples shows about everything you can do, from basic Shell operations to SFTP and much more.
You can locally exec ssh and have it execute the find command on another machine.
e.g.
ssh user#host "find /home/foo -name index.*"
You can exec that from within the JVM and read the output from the process.
When mounting an NFS filesystem, all data handling goes through the nfs client. How can I write my own handlers to use something other than NFS?
An alternative would be a localhost NFS server but that seams awfully inefficient
Edit
Example of what should happen
Normally with a filesystem you get: app reads/writes filesystem, Solaris sees where it is mounted and if it is disk then it reads/writes the disk. If it is software mirror it reads and writes to the mirror software. If it it is NFS it reads and writes to a remote NFS server. I want it to read and write to a custom storage software instead of any of the above mentioned options.
Our storage software is for storing files that applications use, it is geared towards large or frequently replaced chunks of data that are not stored in a database. It also includes certain flexibility specific to our company.
Old/existing applications don't know about our new software. All they know to do is read/write a directory. We could tell Solaris that the directory was hosted on NFS and then the NFS server translates and connects to the storage software. We would prefer to tell Solaris about our new program which Solaris has never heard of and then teach Solaris how to talk to our program.
To me this sounds like you'd have to create a pseudo file system. Solaris uses VFS (Virtual File System), under which you can use different filesystems presented as one uniform structure to userspace. Wheither you mount a UFS or NFS or WHATEVER filesystem, users and applications can use filesystem-agnostic tools to interact with VFS.
That means that what you need to create a pseudo file system; a filesystem that manages to handle the vnode and vfs operations (VFS syscall interface), such as read(), write() etc and tie them, (decide what to do when someone opens a particular file etc), to a database-backend of your choice.
Read more:
http://developers.sun.com/solaris/articles/solaris_internals_ch14_file_system_framework.pdf
Sounds like a big task...
Regards,
jgr
You might want to look at some CIFS servers. Alfresco has JCIFS, which is a CIFS server library in Java. It lets you present resources as files, as if they're on a Windows system. So, that means that programs can "mount" these CIFS servers, and you can publish data from your Database via that mechanism.
I have not used it, but that sounds like what you want to do and perhaps something you may want to look in to.
There's also FUSE which lets you create custom file systems in "user mode" rather than having to hack the kernel. It works on Unix and Mac OS, there may be a Windows version as well. This can, in theory, do anything.
For example, there are instances that let you mount a remote system over SSH using a FUSE system. These tend to be written in C/C++.
NFS isn't about mounting a directory on software but mounting a remote share on a directory. Whether the storage device is remote or not doesn't matter that much, it is still through layers of kernel software. Solaris use VFS to provide the first layer. You should implement the underlying one. That would be quite a difficult task for someone already familiar with VFS. As you obviously are not familiar with writing kernel code, I would be very pessimistic about your project ...
What I would suggest you to do instead would be a simpler and less risky approach. Implement an interposition library that would intercept the application I/O code (open, read, write, close, and the likes or perhaps libc fopen, fwrite, you have to figure out what is the best location to interpose) and call your storage software instead.
Here is a simple example of the process:
http://developers.sun.com/solaris/articles/lib_interposers.html
I have a cluster of 32 servers and I need a tool to distribute a Java service, packaged as a Jar file, to each machine and remotely start the service. The cluster consists of Linux (Suse 10) servers with 8 cores per blade. The application is a data grid which uses Oracle Coherence. What is the best tool for doing this?
I asked something similar once, and it seems that the Java Parallel Processing Framework might be what you need:
http://www.jppf.org/
From the web site:
JPPF is an open source Grid Computing
platform written in Java that makes it
easy to run applications in parallel,
and speed up their execution by orders
of magnitude. Write once, deploy once,
execute everywhere!
Have a look at OpenMOLE: http://www.openmole.org/
This tool enables you to distribute a computing workflow to several resources: from multicores machines, to clusters and computing grids.
It is nicely documented and can be controlled through groovy code or a GUI.
Distributing a jar on a cluster should be very easy to do with OpenMOLE.
Is your service packaged as an EJB? JBoss does a fairly good job with clustering.
Use Bit Torrent. Using Peer to Peer sharing style on clusters can really boost up your deployment speed.
It depends on which operating system you have and how security is setup on your network.
If you can use NFS or Windows Share, I suggest you put the software on an NFS drive which is visible to all machines. That way you can run them all from one copy.
If you have remote shell or secure remote shell you can write a script which runs the same command on each machine e.g. start on all machines, or stop on all machines.
If you have windows you might want to setup a service on each machine. If you have linux you might want to add a startup/shutdown script to each machine.
When you have a number of machines, it may be useful to have a tool which monitors that all your services are running, collects the logs and errors in one place and/or allows you to start/stop them from a GUI. There are a number of tools to do this, not sure which is the best these days.