I would like to stop my current transfer file with using method stopUpload() :
private ChannelSftp channelSftp
private ChannelSftp setupJsch() throws JSchException {
JSch jsch = new JSch();
jsch.setKnownHosts("/Users/john/.ssh/known_hosts");
Session jschSession = jsch.getSession(username, remoteHost);
jschSession.setPassword(password);
jschSession.connect();
return (ChannelSftp) jschSession.openChannel("sftp");
}
public void stopUpload()
{
channelSftp.disconnect();
}
public void whenUploadFileUsingJsch_thenSuccess() throws JSchException, SftpException {
ChannelSftp channelSftp = setupJsch();
channelSftp.connect();
String localFile = "src/main/resources/sample.txt";
String remoteDir = "remote_sftp_test/";
channelSftp.put(localFile, remoteDir + "jschFile.txt");
channelSftp.exit();
}
When stopUpload() run I have this error : Exception in thread "JavaFX Application Thread" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
To cleanly cancel a JSch SFTP transfer, when you need, implement the SftpProgressMonitor interface:
public class CancellableProgressMonitor implements SftpProgressMonitor {
private boolean cancelled;
public CancellableProgressMonitor() {}
public void cancel() {
this.cancelled = true;
}
public bool wasCancelled() {
return this.cancelled;
}
public void init(int op, java.lang.String src, java.lang.String dest, long max) {
this.cancelled = false;
}
public boolean count(long bytes) {
return !this.cancelled;
}
public void end() {
}
}
And pass it to ChannelSftp.put:
CancellableProgressMonitor monitor = new CancellableProgressMonitor()
channelSftp.put(localFile, remoteDir + "jschFile.txt", monitor);
Call monitor.cancel() when you need to cancel the transfer.
public void stopUpload() {
monitor.cancel();
}
If you want to cleanup the partially transferred file:
String remoteFile = remoteDir + "jschFile.txt";
try {
channelSftp.put(localFile, remoteFile, monitor);
} catch (IOException e) {
if (monitor.wasCancelled() && channelSftp.getSession().isConnected()) {
try {
channelSftp.rm(remoteFile);
} catch (SftpException e) {
if (e.id == SSH_FX_NO_SUCH_FILE) {
// can happen if the transfer was cancelled
// before the file was even created
} else {
throw e;
}
}
}
throw e;
}
Related
I'm using JSch for file upload over SFTP. In its current state each thread opens and closes connection when needed.
If it possible to use connection pooling with JSch in order to avoid overhead caused by large number of connection opening and closing?
Here is a example of function called from inside of thread
public static void file_upload(String filename) throws IOException {
JSch jsch = new JSch();
Session session = null;
try {
session = jsch.getSession("user", "server_name", 22);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword("super_secre_password");
session.connect();
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
FileInputStream inputSrr = new FileInputStream(filename);
try {
sftpChannel.put(inputSrr, "/var/temp/"+filename);
} catch (SftpException e) {
e.printStackTrace();
} finally {
if (inputSrr != null) {
inputSrr.close();
}
}
sftpChannel.exit();
session.disconnect();
} catch (JSchException e) {
e.printStackTrace();
} catch (SftpException e) {
e.printStackTrace();
}
}
For that I would prefer commons-pool. ;)
Here's an implementation of Ssh Connection pool
http://www.javacodegeeks.com/2013/02/pool-of-ssh-connections-using-apache-keyedobjectpool.html
you can use grep4j to use this pool
https://code.google.com/p/grep4j/source/browse/trunk/src/main/java/org/grep4j/core/command/linux/SessionFactory.java?r=354
Also make sure you can access the server from the execution machine. For instance if the target server is not in your reach. It'll throw connection timeout.
I wold like to share with you our implementation, We have used Session Manager of jsch-extension library
First of all you need to implement pool object factory that is responsible for lifecycle of pooled objects:
public class ChannelSftpConnectionsFactory extends BasePooledObjectFactory<ChannelSftp> {
private SessionManager sessionManager;
public ChannelSftpConnectionsFactory(final SessionManager sessionManager) {
this.sessionManager = sessionManager;
}
//Create and open channel
#Override
public ChannelSftp create() throws JSchException {
ChannelSftp channelSftp = (ChannelSftp) sessionManager.getSession().openChannel("sftp");
channelSftp.connect();
return channelSftp;
}
//wrapping
#Override
public PooledObject<ChannelSftp> wrap(final ChannelSftp channelSftp) {
return new DefaultPooledObject<>(channelSftp);
}
#Override
//disconnect channel on destroy
public void destroyObject(final PooledObject<ChannelSftp> pooledObject) {
ChannelSftp sftp = pooledObject.getObject();
disconnectChannel(sftp);
}
void disconnectChannel(final ChannelSftp sftp) {
if (sftp.isConnected()) {
sftp.disconnect();
}
}
#Override
//reset channel current folder to home if someone was walking on another folders
public void passivateObject(final PooledObject<ChannelSftp> p) {
ChannelSftp sftp = p.getObject();
try {
sftp.cd(sftp.getHome());
} catch (SftpException ex) {
log.error("Could not reset channel to home folder, closing it");
disconnectChannel(sftp);
}
}
#Override
//validate object before it is borrowed from pool. If false object will be removed from pool
public boolean validateObject(final PooledObject<ChannelSftp> p) {
ChannelSftp sftp = p.getObject();
return sftp.isConnected() && !sftp.isClosed();
}
}
Now you could create pool using configured factory:
ObjectPool<ChannelSftp> createPool(final SessionManager sessionManager, final GenericObjectPoolConfig<ChannelSftp> poolConfig) {
return PoolUtils.synchronizedPool(new GenericObjectPool<>(buildFactory(sessionManager), poolConfig));
}
PooledObjectFactory<ChannelSftp> buildFactory(final SessionManager sessionManager) {
return PoolUtils.synchronizedPooledFactory(new ChannelSftpConnectionsFactory(sessionManager));
}
This java doc would help you to configure pool properly : https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/impl/BaseGenericObjectPool.html
Do not forget about correct borrowing and returning of object into pool: https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/ObjectPool.html
Object obj = null;
try {
obj = pool.borrowObject();
try {
//...use the object...
} catch(Exception e) {
// invalidate the object
pool.invalidateObject(obj);
// do not return the object to the pool twice
obj = null;
} finally {
// make sure the object is returned to the pool
if(null != obj) {
pool.returnObject(obj);
}
}
} catch(Exception e) {
// failed to borrow an object
}
I am attempting to abruptly disconnect from an SFTP connection in Java without using the sftp.disconnect() method. Using this to build an integration test that checks that clean up happens everytime. See the test below:
public void checkDisruptedConnections() throws JSchException, InterruptedException {
ChannelSftp sftp = setupSftp(null);
sftp.connect();
try {
//disrupt connection OVER HERE
} catch (Exception e) {
assertEquals("1", jedis.get(SESSION_KEY));
}
waitForConnectionClose();
assertEquals("0", jedis.get(SESSION_KEY));
}
Ended up using a separate thread that connects and gets interupted.
#Test
public void testSftpInterupt() throws InterruptedException, JSchException, SftpException {
Connect thread = new Connect();
thread.start();
while (thread.isAlive()) {
waitForConnectionClose();
}
waitForConnectionClose();
assertEquals("0", jedis.get(SESSION_KEY));
}
Connect looks as follows:
private class Connect extends Thread {
#Override
public void run() {
ChannelSftp sftp = null;
Thread thread = this;
SftpProgressMonitor monitor = new SftpProgressMonitor() {
#Override
public void init(int op, String src, String dest, long max) {
}
#Override
public boolean count(long count) {
currentThread().interrupt();
return false;
}
#Override
public void end() {
}
};
try {
sftp.connect();
sftp.put(LOCAL_FILE_LARGE, remoteFile, monitor);
} catch (JSchException | SftpException e) {
assertEquals("java.io.InterruptedIOException", e.getMessage());
}
}
}
This is the SocketServer code that generates a server thread
public class ProcessorCorresponder {
protected final static Logger logger = LogManager.getLogger( ProcessorCorresponder.class );
private static int port = Integer.parseInt(PropertiesLoader.getProperty("appserver.port") == null ? "666" : PropertiesLoader.getProperty("appserver.port"));
private static int maxConnections = Integer.parseInt(PropertiesLoader.getProperty("appserver.maxconnections") == null ? "666" : PropertiesLoader.getProperty("appserver.maxconnections"));
public static void main(String[] args) {
logger.info("Starting server .. "
+ "[port->" + port
+ ",databaseName->" + databaseName + "]");
try (ServerSocket listener = new ServerSocket();) {
listener.setReuseAddress(true);
listener.bind(new InetSocketAddress(port));
Socket server;
int i = 0;
while((i++ < maxConnections) || (maxConnections == 0)) {
server = listener.accept();
logger.debug(
"New Thread listening on " + server.getLocalAddress().toString() + ":" + server.getLocalPort()
+ ", initiated from IP => " + server.getInetAddress().toString() + ":" + server.getPort()
);
MySocketServer socSrv = new MySocketServer (server);
Thread t = new Thread( socSrv );
t.start();
}
} catch (Exception ex) {
logger.error("Error in ProcessorInterface", ex);
}
}
}
Server code: This is a thread to handle one connection, there is a program that monitors a serversocket and spins off request threads as needed.
public class MySocketServer implements Runnable {
protected final static Logger logger = LogManager.getLogger(MySocketServer.class);
private final Socket server;
// because we are using threads, we must make this volatile, or the class will
// never exit.
private volatile boolean shouldContinue = true;
private StringBuffer buffHeartbeatMessage = new StringBuffer().append((char) 0).append((char) 0).append((char) 0)
.append((char) 0).append((char) 0).append((char) 0);
private Heartbeat heartbeat = new Heartbeat(/* 60 */3000, buffHeartbeatMessage.toString());
public MySocketServer(Socket server) {
this.server = server;
}
#Override
public void run() {
try (BufferedReader in = new BufferedReader(new InputStreamReader(this.server.getInputStream()));
BufferedOutputStream out = new HeartbeatBufferedOutputStream(this.server.getOutputStream(),
heartbeat)) {
final StreamListener listener = new StreamListener(in);
listener.addActionListener(new ActionListener() {
#Override
public void actionPerformed(ActionEvent event) {
if (event.getID() == ActionEvent.ACTION_PERFORMED) {
if (event.getActionCommand().equals(StreamListener.ERROR)) {
logger.error("Problem listening to stream.");
listener.setShouldContinue(false);
stopRunning();
} else {
String messageIn = event.getActionCommand();
if (messageIn == null) { // End of Stream;
stopRunning();
} else { // hey, we can do what we were meant for
logger.debug("Request received from client");
// doing stuff here
...
// done doing stuff
logger.debug("Sending Client Response");
try {
sendResponse(opResponse, out);
} catch (Exception ex) {
logger.error("Error sending response to OP.", ex);
}
}
}
}
}
});
listener.start();
while (shouldContinue) {
// loop here until shouldContinue = false;
// this should be set to false in the ActionListener above
}
heartbeat.setShouldStop(true);
return;
} catch (Exception ex) {
logger.error("Error in ESPSocketServer", ex);
return;
}
}
private void stopRunning() {
shouldContinue = false;
}
private void sendResponse(ClientResponse opResponse, BufferedOutputStream out) throws Exception {
logger.debug("Before write");
out.write(opResponse.getResponse().getBytes());
logger.debug("After write. Before flush");
out.flush();
logger.debug("After flush");
// this log message is in my logs, so I know the message was sent
}
}
My StreamListener class.
public class StreamListener extends Thread {
protected final static Logger logger = LogManager.getLogger(StreamListener.class);
public final static String ERROR = "ERROR";
private BufferedReader reader = null;
private List<ActionListener> actionListeners = new ArrayList<>();
private boolean shouldContinue = true;
public StreamListener(BufferedReader reader) {
this.reader = reader;
}
#Override
public void run() {
while (shouldContinue) {
String message;
try {
// client blocks here and never receives message
message = reader.readLine();
ActionEvent event = new ActionEvent(this, ActionEvent.ACTION_PERFORMED, message);
fireActionPerformed(event);
} catch (IOException e) {
e.printStackTrace();
ActionEvent event = new ActionEvent(this, ActionEvent.ACTION_PERFORMED, ERROR);
fireActionPerformed(event);
}
}
}
public void setShouldContinue(boolean shouldContinue) {
this.shouldContinue = shouldContinue;
}
public boolean getShouldContinue() {
return shouldContinue;
}
public boolean addActionListener(ActionListener listener) {
return actionListeners.add(listener);
}
public boolean removeActionListener(ActionListener listener) {
return actionListeners.remove(listener);
}
private void fireActionPerformed(ActionEvent event) {
for (ActionListener listener : actionListeners) {
listener.actionPerformed(event);
}
}
}
My Heartbeat class
public class Heartbeat extends Thread {
private BufferedOutputStream bos = null;
private int beatDelayMS = 0;
private String message = null;
private boolean shouldStop = false;
public Heartbeat(int beatDelayMS, String message) {
this.beatDelayMS = beatDelayMS;
this.message = message;
setDaemon(true);
}
#Override
public void run() {
if (bos == null) { return; }
while(!shouldStop) {
try {
sleep(beatDelayMS);
try {
bos.write(message.getBytes());
bos.flush();
} catch (IOException ex) {
// fall thru
}
} catch (InterruptedException ex) {
if (shouldStop) {
return;
}
}
}
}
public void setBufferedOutputStream(BufferedOutputStream bos) {
this.bos = bos;
}
public BufferedOutputStream getBufferedOutputStream() {
return bos;
}
public void setShouldStop(boolean shouldStop) {
this.shouldStop = shouldStop;
}
public boolean getShouldStop() {
return shouldStop;
}
}
My HeartbeatBufferedOutputStream
public class HeartbeatBufferedOutputStream extends BufferedOutputStream {
private Heartbeat heartbeat = null;
public HeartbeatBufferedOutputStream(OutputStream out, Heartbeat heartbeat) {
super(out);
this.heartbeat = heartbeat;
this.heartbeat.setBufferedOutputStream(this);
heartbeat.start();
}
#Override
public synchronized void flush() throws IOException {
super.flush();
heartbeat.interrupt();
}
}
And finally here is the "Client" class
public class Mockup extends Thread {
protected final static Logger logger = LogManager.getLogger(Mockup.class);
// because we are using threads, we must make this volatile, or the class will
// never exit.
private volatile boolean shouldContinue = true;
public static void main(String[] args) {
new Mockup().start();
}
#Override
public void run() {
try (Socket socket = new Socket("localhost", 16100);
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));) {
final StreamListener listener = new StreamListener(in);
listener.addActionListener(new ActionListener() {
#Override
public void actionPerformed(ActionEvent event) {
if (event.getID() == ActionEvent.ACTION_PERFORMED) {
if (event.getActionCommand().equals(StreamListener.ERROR)) {
logger.error("Problem listening to stream.");
listener.setShouldContinue(false);
stopRunning();
} else {
String messageIn = event.getActionCommand();
if (messageIn == null) { // End of Stream;
stopRunning();
} else { // hey, we can do what we were meant for
// convert the messageIn to an OrderPower request, this parses the information
logger.info("Received message from server. [" + messageIn + "].");
}
}
}
}
});
listener.start();
StringBuffer buff = new StringBuffer("Some message to send to server");
logger.info("Sending message to server [" + buff.toString() + "]");
out.write(buff.toString().getBytes());
out.flush();
boolean started = false;
while (shouldContinue) {
if (!started) {
logger.debug("In loop");
started = true;
}
// loop here until shouldContinue = false;
// this should be set to false in the ActionListener above
}
logger.info("Exiting Mockup");
return;
} catch (Exception ex) {
logger.error("Error running MockupRunner", ex);
}
}
private void stopRunning() {
shouldContinue = false;
}
}
I have confirmed from logging messages that the Server sends a message to the BufferedOutputStream, and is flushed, but the Client logs indicate that it is blocked on the reader.readLine() and never gets the message.
You are reading lines but you are never writing lines. Add a line terminator to what you send.
I´m try to build a application which, by threads, are listening to directory for changes. Everything works fine, but there is a little bug. I´m very new to the whole threads-thing... So, this problem probably based on my ignorance...
The program can se all the changes in the picked directory, BUT, when the threads are running, i cant modify the files inside the directory... Those are locked in the process...
I will be very happy if someone perhaps can give me some tips in how i can solve this.
Thank you in advance
DirWatch
public class DirWatch implements Runnable{
Path dirPath;
private boolean run = true;
private boolean created = false;
private boolean modified = false;
private boolean compressed = false;
private boolean isJSON = false;
/**
*
* #param path
* #throws IOException
*/
public void setWatchPath(String path) throws IOException {
dirPath = Paths.get(path);
try {
Boolean isFolder = (Boolean) Files.getAttribute(dirPath, "basic:isDirectory", new LinkOption[]{NOFOLLOW_LINKS});
if (!isFolder) {
throw new IllegalArgumentException("Path: " + dirPath + " is not a folder");
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
System.out.println("Watching path: " + path);
}
public void setDirPathWatchRun(boolean run){
this.run = run;
}
public boolean isCreated() {
return created;
}
public void setCreated(boolean created) {
this.created = created;
}
public boolean isModified() {
return modified;
}
public void setModified(boolean modified) {
this.modified = modified;
}
public boolean isCompressed() {
return compressed;
}
public void setCompressed(boolean compressed) {
this.compressed = compressed;
}
public boolean isJSON() {
return isJSON;
}
public void setJSON(boolean JSON) {
isJSON = JSON;
}
private void checkFileType(String fileName){
String extension = fileName.substring(fileName.length() - 4);
if(extension.equalsIgnoreCase(FILE_TYPE.TAR.getFileType())){
setCompressed(true);
System.out.println(extension);
}
if(extension.equalsIgnoreCase(".json")){
setJSON(true);
}
}
#Override
public void run() {
FileSystem fs = dirPath.getFileSystem ();
try(WatchService service = fs.newWatchService()) {
dirPath.register(service, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
WatchKey key = null;
while(run) {
key = service.take();
WatchEvent.Kind<?> kind = null;
for(WatchEvent<?> watchEvent : key.pollEvents()) {
kind = watchEvent.kind();
if (OVERFLOW == kind) {
System.out.println("OVERFLOW");
continue;
} else if (ENTRY_CREATE == kind) {
System.out.println("New path created: " + watchEvent.context().toString());
setCreated(true);
checkFileType(watchEvent.context().toString());
} else if (ENTRY_DELETE == kind){
System.out.println("Path deleted: " + watchEvent.context().toString());
setModified(true);
checkFileType(watchEvent.context().toString());
} else if (ENTRY_MODIFY == kind) {
System.out.println("Path modified: " + watchEvent.context().toString());
setModified(true);
checkFileType(watchEvent.context().toString());
}
}
if(!key.reset()) {
break;
}
}
} catch (InterruptedException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Method Watch in another class
private void watch() throws IOException {
watchJSON = new DirWatch();
watchTAR = new DirWatch();
watchTAR.setWatchPath(serverArgs.getCompressedPath());
watchJSON.setWatchPath(serverArgs.getExtractedPath());
Runnable checkSourceActions = new Runnable() {
#Override
public void run() {
while(true) {
if (watchJSON.isCreated() || (watchJSON.isModified())) {
server();
}
if(watchTAR.isCreated() || (watchTAR.isModified())) {
extractFiles(fileHandler);
createManifest(fileHandler);
server();
}
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
};
Thread thread1 = new Thread(watchJSON);
Thread thread3 = new Thread(watchTAR);
Thread thread2 = new Thread(checkSourceActions);
thread1.start();
thread2.start();
thread3.start();
}
When I try to change the file while the program is running
I need to execute(run) a shell script which resides in the server(Solaris) from java. Please help me how to execute a file from java.? I have tried with sendCommand() of TelnetToClient. So please help me in running a file from my GUI.
The program goes like this.
TelnetToPort tele = new TelnetToPort("opmer3");
tele.login("root","root");
String command_ = "/usr/bin/bash /opt/nrl/logs/applications/ns/lccommands.sh";
tele.runComm(command_);
If you are looking for optimized solution for executing any scripts for your java class, then you can use Jsch with Google Expect4j libraries.
For jsch, go to http://www.jcraft.com/jsch/
For Expect4j, go to http://code.google.com/p/expect4j/
Following is small code sample for log in and executing file fro remote java class.
private Expect4j SSH(String hostname, String username,String password, int port) throws Exception {
JSch jsch = new JSch();
Session session = jsch.getSession(username, hostname, port);
if (password != null) {
session.setPassword(password);
}
Hashtable<String,String> config = new Hashtable<String,String>();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect(60000);
channel = (ChannelShell) session.openChannel("shell");
Expect4j expect = new Expect4j(channel.getInputStream(), channel.getOutputStream());
channel.connect();
return expect;
}
This method will open up SSH stream to remote server which will be used by expect4j for sending commands.
private boolean executeCommands() {
boolean isSuccess = true;
Closure closure = new Closure() {
public void run(ExpectState expectState) throws Exception {
buffer.append(expectState.getBuffer());
expectState.exp_continue();
}
};
List<Match> lstPattern = new ArrayList<Match>();
String[] regEx = SSHConstants.linuxPromptRegEx;
if (regEx != null && regEx.length > 0) {
synchronized (regEx) {
for (String regexElement : regEx) {//list of regx like, :>, /> etc. it is possible command prompts of your remote machine
try {
RegExpMatch mat = new RegExpMatch(regexElement, closure);
lstPattern.add(mat);
} catch (MalformedPatternException e) {
return false;
} catch(Exception e) {
return false;
}
}
lstPattern.add(new EofMatch( new Closure() { // should cause entire page to be collected
public void run(ExpectState state) {
}
}));
lstPattern.add(new TimeoutMatch(defaultTimeOut, new Closure() {
public void run(ExpectState state) {
}
}));
}
}
try {
Expect4j expect = SSH(objConfig.getHostAddress(), objConfig.getUserName(), objConfig.getPassword(), SSHConstants.SSH_PORT);
expect.setDefaultTimeout(defaultTimeOut);
if(isSuccess) {
for(String strCmd : lstCmds)
isSuccess = isSuccess(lstPattern,strCmd);
}
boolean isFailed = checkResult(expect.expect(lstPattern));
return !isFailed;
} catch (Exception ex) {
return false;
} finally {
closeConnection();
}
}
private boolean isSuccess(List<Match> objPattern,String strCommandPattern) {
try {
boolean isFailed = checkResult(expect.expect(objPattern));
if (!isFailed) {
expect.send(strCommandPattern);
expect.send("\r");
return true;
}
return false;
} catch (MalformedPatternException ex) {
return false;
} catch (Exception ex) {
return false;
}
}
Hope this help.
Thanks.
new ProcessBuilder("ssh", "root#opmer3", command_).start();