I need help regarding how to Automate download logs from server via ssh file transfer to local machine.Currently I am downloading logs via login to ssh tool and select the path of the logs and select all the logs and then drag and drop them to local drive but this takes time on VPN.
Is there any way to create batch file or run any Java program to downloads all logs just in one line command or single execution automatically.I have tried to check this google but with scp and ftp command I am not able to proceed further.Please suggest.
If you're on Windows, I'd recommend WinSCP's easy scripting.
Just write your script of where to connect, where to navigate to, what to collect and where to copy your file in your machine. You can even copy to another server. All in one script. Examples are easy to follow in the link.
You can also run a simple script that does the work in one line:
winscp.exe /console /command "option batch on" "open rahul#logsserver.com" "get 02012011logs.txt d:\" "exit"
What is the OS you are using for your client? If your client machine is *nix variant, you could write a simple bash script, which when paired with SSH Keys, will allow you your one command downloader.
For a windows variant, you could do similar with a batch script, schedule tasks and pscp w/ pagent saving a private key in memory.
Of course none of this deals with starting up or tearing down the vpn.
Write a batch file that uses pscp to download the files you need. Set up public-key authentication to skip password entry. Enable ssh compression for faster download. Automate it via cron. Theoretically one should be able to use the vpn only for the batch file, practically very difficult to set up that way.
The best option for you would be to write a cron job that copies the log using scp on the regular intervals you want.
However, if you do want to do it in Java. Go for JSch.
Side-note: none of the options above gets you rid of the VPN. You will have to be on VPN to get file transferred, if it's required. So, stay connected to VPN while you are transferring your files in whatever way you like.
Edit
You should update your question with the platform that you're using. My previous answer was assuming that you're on Unix based.
If you are using Windows, you can write a batch script that uses PSCP to transfer the files to your local machine. You can set it as Scheduled Task in Windows to run periodically.
Related
I made a simple chat room application consists of a client.jar and server.jar. After running the .jar file in cmd window, I need to type in some specified command in client part and carriage return to run the application, such as "#connect": connect to the server, "#send XXX": send XXX message to server and broadcast to other connected clients...
Here comes the problems. Actually I want to test how the application works well when there exist concurrency. So I think I need try to let two or more client parts send the command repeatedly and fast. But obviously i cannot achieve manually.
Is there any ways can let the system get the command and send it automatically at high speed? I guess using batch file is suitable for this issue, but I am not sure or how to implement it...
I am using Bamboo from Atlassian as CI tool. I have a requirement where I need to create a new plan which helps in moving files from one server to another server.
Why I need to do this?
One of the server is our production server and we don't want it to contain any junk data but at the same time we don't want to delete any file, so as a solution we will move the files into different server.
Could anyone please help me on this?
We had to do something similar and ended up using a simple bash script for it (we were dealing with several directories full of files).
Bamboo has a SCP task:
Using the SCP task in Bamboo
Use SSH task to login to prod server and execute mv commands on it
I need to move few files in a directory to another directroy in another machine. The source and destination directory names are known in advance. These directories will be in different Operating System(i.e. the source can be Linux and Destination can be in Windows).
To solve this, i wrote a Socket program in Java that successfully copies files from source directory to a destination directory(Tested in Windows OS). But this application needs program running in both the host and remote machine. So, my question is :
1) Is there any way to transfer files(in the ways described above) using Java Socket program without having a program running in the remote machine?
2) Whether file transfer using Java socket program, bypass firewal. In other words, if a firewall is present in the remote machine will that stop the transfer process?
3) Any alternative approach to transfer files(in the ways described above) using Java where only the host or remote need to run.?
Thanks in advance,
nks
1) You always need a program running on both machines. However, a way to get around this problem is to use a program already running on one of these machines.
e.g. If you make the directory a Windows share (assuming this is already running) you can mount this share on the Linux box (using smbfs), you can then copy files using cp This means you don't need Java on either end, but you need A program and A service (but it can be an existing one)
2) There isn't a way to by-pass a firewall unless your network is seriously mis-configured. This is the whole point of having one.
3) Once you mount either the Windows box on Linux, or the Linux files (using Samba) on Windows youc an copy files easily, even using Java. ;)
I'm currently doing a project that has to be run on a windows machine. The application creates a CSV file and saves it to the windows filesystem. I would like a way to transfer this file to a machine running linux into a directory selected by the user.
I am not asking for code (although if it helps then feel free :P ) but asking more for what protocols to use (ftp etc) and wether I need to take anything into account such as permissions in Linux as I am fairly in experienced with programming for linux file systems.
Thanks in advance :)
Can the windows machine see the linux filesystem? Is there a samba server running on the linux machine? If so, you can simply copy it from one filesystem to the other.
If not, you can try SCP or SFTP but you need to have the keys setup. Another way is to use Runtime.exec to copy the file using a windows specific command utility.
If all else fails, run a java process on the linux server and connect to it using sockets. Your windows client can write the file to the socket and it can be written out by the linux server.
I would try using SSH as SCP is a fairly standard and convenient way to transfer files securely. In Java you can use jssh amoungst other libraries.
If you don't want to do it programatically, I'd suggest using Rsync. It should be available for most *nix distros and you've got a Windows version called DeltaCopy that implements the same protocol.
Scenerio: I'd like to run commands on remote machines from a Java program over ssh (I am using OpenSSH on my development machine). I'd also like to make the ssh connection by passing the password rather than setting up keys as I would with 'expect'.
Problem: When trying to do the 'expect' like password login the Process that is created with ProcessBuilder cannot seem to see the password prompt. When running regular non-ssh commands (e.g 'ls') I can get the streams and interact with them just fine. I am combining standard error and standard out into one stream with redirectErrorStream(true); so I am not missing it in standard error...When I run ssh with the '-v' option, I see all of the logging in the stream but I do not see the prompt. This is my first time trying to use ProcessBuilder for something like this. I know it would be easier to use Python, Perl or good ol' expect but my boss wants to utilize what we are trying to get back (remote log files and running scripts) within an existing Java program so I am kind of stuck.
Thanks in advance for the help!
The prompt might only be shown when ssh is connected to a TTY, which it isn't in the case of Java.
There's probably a way to supply the password on the command-line in your ssh application. That will be the way to get past the prompt.
Alternately, consider connecting directly to the host server from native Java code rather than running an external application. There's a million libraries that will do this.
Rather than using an external ssh program, why not use a Java ssh library:
Trilead
JTA
Are two I found with google - that'll avoid the problem that openssh will be working very hard to prevent entering the password on stdin - it'll be opening the terminal directly. expect has to work very hard to simulate a tty in order to work.
Why not use a Java ssh client? This one is BSD-licensed, and there are more clients listed here.
Most security minded programs don't use stdin/stdout for capturing passwords, they capture the TTY or some equivalent method.
Echoing others' suggestion to use a Java SSH library. But wanted to comment on Cohen's response. Sending your password over the command line when establishing the connection is insecure and also not permitted by many sshd servers (based on configuration).
You might want to look into setting up keys for this, so you can perform ssh commands between the machines without a password.
Basic steps
- use openssh to create a keypair (I've done RSA but I know there's a better method now)
- create a .ssh directory in your home folder on the SOURCE machine
- create a .ssh directory in your home folder on the TARGET machine
- keep your private key in your source machine's .ssh folder
- copy your public key into a file called authorized_keys in the target's .ssh folder
Some instructions can be found here
You can run commands using edtFTPj/PRO, as well as performing file transfers via SFTP. It's Java.