I have a Java app running on a remote server (debian linux). The app logs runtime informations to files.
Once in a while, the support needs to analyze the logs using a client tool (not running on the server, but on the support's desktop). The tool then needs remote access to
old log files on the server
the current log file (growing in real time)
I'm searching a good (=standard, flexible, secure, well debugged etc) way to stream these past and present log messages to my client tool.
How would go about doing this?
Does a library exist, which provides this service?
Can syslog be adapted to do this?
Edit: For privacy reasons, the log files should be stored on the remote server only and not generally sent to a central logging service.
Why not use Log4j's capability as a publisher of log files to a remote server via TCP? Log4j is a standard Java logging tool (well-known/well-documented) and setting up the remote connectivity is pretty straightforward.
I've used this in the past to transmit across networks and collate log files from multiple machines in one common searchable directory structure (very useful in grid environments when you don't know where your processes could be running)
See this page for more details.
The simplest approach is likely to setup samba on the Linux box so you can access the logs remotely as they are updated without having to copy them.
You could use rsync. This allows you to securely copy changes to files between machines. i.e. an appending file would only copy the changes since the last rsync. Note: you have to run it repeatedly to keep seeing updates.
If the tools could be modified to support SSH e.g. via VFS, then you could access the remote files as if they were local.
Related
Hi I have been doing some manual tasks which consume some of my time and I want to automate it the tasks are:
Connecting to remote windows server using mstsc command and restarting some services.
Connecting to remote windows server and modifying the files, checking the modifications and then again reverting the changes when the changes are tested.
I want to know whether I can achieve a one click solution for this scenario by writing some code in java and reducing the manual time.
Or is there any other solution for the same which can be generic and could be implemented on other servers too.
The steps for the solution to cover would be:
Connect to remote machine using username and password.
Restarting the services from the code or just executing a batch file for the same which could be lying in some folder on the same machine.
Modifying some files on the remote machine.
I have a Spring web application which triggers a SAS job on a remote Linux server, the SAS job will generate a result file on the remote server upon finished. I need to display the result on my Spring application, so I want to create a listener for the directory changes on the server.
I have being looking at the java.nio library, but it looks like it only works local directories. Any ideas other than keep pinging the server through ssh? Thanks!
You might use FTP from org.apache.commons.net.ftp
Using FTP (or any other Java FTP library), you only need to check for the content on the remote directory.
If the directory supposedly is always empty, then when the first file appears your process will be triggered.
If the directory is not always empty, you might need to implement something to control which files are new, and which are not.
Please let me know if you need further assistance.
VisualVM/VisualGC provides quite a lot of useful GC-related metrics in real-time, but I want a command-line tool that can connect to a remote application via jstatd and record in CSV or XML ideally exactly the same metrics as VisualGC provides. It shouldn't be a problem to write my own, but from the first look I cannot find what protocol visualvm/jstatd use.
As I understand VisualGC is not open source: https://stackoverflow.com/questions/11096466/where-is-the-source-code-repository-for-visualgc, but are there any alternative open source tools? Are there any clues how about the protocol used?
I managed to connect to jstatd via RMI (default port 1099, JStatRemoteHost is the rmi-name), can attach to a particular VM - but http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/jvmstat/monitor/remote/RemoteVm.java#RemoteVm is rather a very low-level interface with jstatd.
I've checked out the VisualVM source code from https://svn.java.net/svn/visualvm~svn/trunk
and just started looking at the 5mb codebase, however it doesn't use neither RemoteHost nor RemoteVm RMI-classes.
Thanks.
First I looked at the source code of VisualVM (~5mb), created a command-line tool that registers JvmstatModelProvider on JvmstatModelFactory, gets an Application, creates JvmJvmstatModel and extracts all MonitoredValues - it connects to a remote application via jstatd and prints changed values in format timeMillis,name,value
1369270235646,sun.gc.generation.2.space.0.capacity,16777216
1369270236666,sun.os.hrt.ticks,2511500491
1369270237581,sun.gc.generation.0.space.0.used,641408
1369270237582,sun.os.hrt.ticks,2512502544
One hour later I found the right command-line tool for my task https://code.google.com/p/hatter-source-code/wiki/hotstat, that doesn't use jstatd - but fine, I can still run it remotely via ssh.
I believe build a small home-made program to make ourselves more comfortable is quite common nowadays. Just few days before, I really tired to get the same named log files from different remote devices through FTP connection again and again so that I started to build one Java web application.
The purpose of the Java web application is simple as once the user filled in the absolute path of source file in remote device and selected corresponding remote devices he or she want to connect to, the web application will finally store those same named log files in user's local computer with well organized folder structure. You can simply understand that this Java servlet is a proxy sits between client and remote devices.
Currently, I have already done and tested the downloading function from remote devices to the server in Java servlet by using Apache common net FTPClient library. It worked fine and provided me the copies of same named log files in a well organized folder structure.
However, when I moved on, I realized that the "pushing" function maybe the killer. Following are few queries I want to discuss with you all:
Even I could get IP address or host name from client's requests, is it possible or suitable for me to auto establish a FTP connection from servlet to client?
If an auto FTP connection is achievable, what are the security concerns I should pay attention?
If an auto FTP connection is not achievable, is it possible or suitable for me to return those files in the response to the client?
I appreciate your comments or suggestions. Hope you all also enjoy the open discussion here.
I want to access to a file (read) that exist on a remote machine from my code JAVA,what I need to do that?
just the IP of the machine and the location of the file or I need somthing else?
Thank you
There are some choices:
Via a 'mapped' directory using SMB/Samba to the remote machine and you can then access the file using the normal File class.
Via a Web Server where read access is easier (if you require write access then you are looking at something like WebDAV). This requires the use of the HTTP protocol in your code.
Via FTP or SFTP network protocols to access the file. This obviously requires the use of (S)FTP classes to access the file.
The first option is easiest from a coding point-of-view.
If both the Java code and the remote file are on Linux machines, you can also choose NFS.
As always you need to start a server which serves the file - you need nfsd to share the directory containing that file on the remote machine.
On the machine where your Java code will run, mount the shared nfs
Here is a brief introduction of using nfs on Ubuntu.
If you prefer FTP/HTTP, you will be interested in Apache commons vfs library, which supports many protocols including FTP, SFTP, HTTP, etc.
First of all, you need a service on the remote machine that serves files. Once a file-serving service exists, you communicate with the service using its protocol.
Assuming the client-server model, you have several choices on the remote (server) side. First of all, you can design your own protocol, write a server, deploy it on the remote machine and write a client (in Java) which will talk with the server using the designed protocol. However, there many off-the-shelf solutions (protocols + servers + Java client libraries) that might be used. Three protocols which come to mind right now: TFTP, FTP and SMB.
If your aim is simplicity, I recommend TFTP: there are free TFTP servers for UNIX, Windows and Mac OS X, and there is Apache Commons Net Java library on the client side.