I'm using Apache Commons VFS / SFTP, we are trying to download files from the IBM MVS system.
The download part is all good, however, we can not open up the zipped files after downloading. Seems like the zip file was compressed using a different algorithm or something
Anyone has any pointers?
*Note, the same function works fine if we connect to a regular unix/linux SFTP server.
Below is an example of what we did
String defaultHost = "[my sftp ip address]";
String host = defaultHost;
String defaultRemotePath = "//__root.dir1.dir2.";
String remotePath = defaultRemotePath;
String user = "test";
String password = "test";
String remoteFileName = "Blah.ZIP.BLAH";
log.info("FtpPojo() begin instantiation");
FileObject localFileObject = fsManager.resolveFile("C:/Work/Blah.ZIP.BLAH");
log.debug("local file name is :"+localFileObject.getName().getBaseName());
log.debug("FtpPojo() instantiated and fsManager created");
String uri = createSftpUri(host, user, password) + ":322"+remotePath+remoteFileName;
remoteRepo = fsManager.resolveFile(uri, fsOptions);
remoteRepo.copyFrom(localFileObject, Selectors.SELECT_ALL);
I am not a mainframe expert but I assume you connect not to MVS but to the USS (Unix System Services) which run either FTP or SSH daemon.
I only tried to download files via FTP from USS with a regular Windows FTP could exchange and open the files fine. No binary thouhg.
Thus, I do not assume that you would get files in EBCDIC encoding.
Have you tried downloading a simple text file?
Have you tried downloading a text file with special characters? e.g. German umlauts or other non-ASCII?
A couple things to try:
specify the BINARY option as I think the default is EBCDIC-ASCII translation. This will do horrible things to a zip file. If you can download a plain text file that may be your trouble.
Could the zipped data be something your system doesn't like such as a PDS?
have a read of this page http://www.cbttape.org/downloadtrouble.htm
Related
I'm trying to save the edited PDF which I fetched from the remote server back to its location without having it downloaded/stored on the local machine. I'm using JSch SFTP method to get the input PDF file from the SFTP server using
x = new BufferedInputStream(channelSftp.get("example.pdf"));
PDDocument document = PDDocument.load(x);
and after doing some edits using PDFbox, I'm trying to save it using:
documents.save(new File("ftp/path/location"));
I am not able to because I know it only works for your local directory only. Also I can see that document.save accept OutputStream` parameter, but I do not know how to use it here.
I don't have any problems with taking input using stream reader.
All I need is to save that edited PDF back to its location (possibly replace) without having to download it on my local system.
Use the ChannelSftp.put overload that returns OutputStream:
try (OutputStream out = channelSftp.put("example.pdf")) {
documents.save(out);
}
I'm using apache's commons-net 3.5 to upload files to a remote FTP server, and setting the connection encoding to UTF-8 like below, before openning the connection.
ftpClient.setAutodetectUTF8(true);
ftpClient.setControlEncoding("UTF-8");
And this is the part that sends the file
private void uploadFile(byte[] data, String path, String fileName, FTPClient ftpClient) throws IOException {
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
ftpClient.setFileTransferMode(FTPClient.BINARY_FILE_TYPE);
boolean uploadSuccess = ftpClient.storeUniqueFile(fileName, new ByteArrayInputStream(data));
Here is the problem; when sending files whose name contains Turkish characters, the result file is named incorrectly. Like,
Kimlikş.pdf -> KimlikÅ.pdf
But if I use ftp4j, everything works fine. Did anyone have this before? The data ships incrorrectly from my side. I monitored the traffic with Microsoft Network Monitor. Here is the raw request:
FTP FTP:Request from Port 62642,'STOR KimlikÅ.pdf' {TCP:5879, IPv4:134}
try to use WinSCP - it was working like a charm.
https://winscp.net/eng/download.php
no FileZilla, no TotalCommander - only WinSCP.
and by copy again (for example I had about 6k files and only 50 with cyrilic) only give at question prompt "no to all" and WinSCP copies only left files. great !
5 stars to the free software !
I guess there was a bug with apache's commons-net, so I migrated to ftp4j and the problem was no more.
<dependency>
<groupId>it.sauronsoftware</groupId>
<artifactId>ftp4j</artifactId>
<version>1.6</version>
</dependency>
OS: macbook
I wanna read remote file in Java.
I test it on my local computer first.
I tried to use File, but you guys told me File can only be used on local file system, thanks.
public static void main(String[] args) throws URISyntaxException {
URI uri = new URI("file://127.0.0.1/Users/jian/Public/logfiles/Hadoop/hdp_log1.log");
File file = new File(uri);
System.out.println(file.exists());
}
I find a library for Macbook, Samba JCIFS, from this link access to file using Java with Samba JCIFS . Here is my code :
public static void main(String[] args) throws URISyntaxException, MalformedURLException, SmbException {
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication("","jian", "ywnk");
SmbFile smbFile = new SmbFile("smb://127.0.0.1/Users/jian/Public/logfiles/Hadoop/hdp_log1.log",auth);
System.out.println(smbFile.exists());
}
Then I got this error:
Exception in thread "main" jcifs.smb.SmbAuthException: Logon failure: unknown user name or bad password.
username is the same as I run who am i in terminal, and password is my login password. I don't know why I get this username or password incorrect error.
Thanks for your guys help.
File cannot be used to read "remote files". It is meant to be used for filesystem objects which are only local to the machine, or to be more precise, accessible from the local filesystem.
It so happens that with Unix systems you can have mount points to remote filesystems, for instance an NFS mount, or a FUSE mount. But those will always be seen as "local".
If what you want is to access a remote resource to be viewed as a "file", then File can't do it.
But JSR 203 can. See here for an example.
Therefore, you first need to define what you mean by a "remote file". It entirely depends on the protocol you use. And File cannot do it. Unless the OS has sorted it for you (that is, as mentioned above, you have an NFS mount, a FUSE mount or other).
As to your error, the File constructor warns you that there is an authority component in your URI; and there is: 127.0.0.1. As File expects nothing but a path component to the URI (in addition to the URI scheme, file), this explains the error.
Your requirement can easily be fulfilled by some java FTP library.
Take a look at ftp4j, its a good library for upload and download of files and also, for directory listings.
Apache commons-net is also a good option. They have some really good sample programs too.
My goal is to to convert a .txt file on an FTP page to a simple String for easy manipulation.
The specific .txt file is here: ftp://ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt. It is an anonymous FTP page, so when I use my computer's browser, there's no need for a username or password.
I've tried incorporating different codes and tips from the following sources:
Reading Text File From Server on Android
http://examples.javacodegeeks.com/core-java/apache/commons/net-commons/download-file-from-ftp-server/
How to read a text file via FTP?
Reading txt file from an ftp server and returning it from AsyncTask
unable to read file from ftp in android?
Howto do a simple ftp get file on Android
http://www.javaworld.com/article/2073325/java-app-dev/java-ftp-client-libraries-reviewed.html
http://developer.android.com/reference/java/net/URLConnection.html
None of what I've tried above helped. I'm not quite sure what I'm doing wrong.
To be clear, all I want to do is to get the plain text from the posted .txt file. I have no interest in downloading said file onto my device's memory.
If you could provide me with a step-by-step explanation on how to do this, I'd be very thankful.
Ok. I've got it. For those who are in the same boat, the step-by-step answer is below:
A lot of problems other users were encountering could be solved by having permissions for internet turned on in the manifest, but mine was a little more complicated. Turns out, the main trick is to not include ftp:// in the address in Java.* Also, when you are entering an FTP site, make sure you enter via the root page, so my original page of ftp://ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt becomes: ftp.nasdaqtrader.com.
Make sure to download and include the right Apache Commons library in your project (here: http://commons.apache.org/proper/commons-net/download_net.cgi).
Connect to the root page:
FTPClient ftpClient = new FTPClient();
ftpClient.connect("ftp.nasdaqtrader.com");
Login anonymously, in this case, both username and password are "anonymous". This might be the case with many FTP pages, but I can't be sure:
ftpClient.login("anonymous", "anonymous");
Then, go to the correct directory (be careful about including/excluding the slashes):
ftpClient.changeWorkingDirectory("/SymbolDirectory");
Finally! You can now get the InputStream from the posted text file:
InputStream is = new BufferedInputStream(ftpClient.retrieveFileStream("nasdaqlisted.txt"));
Then, convert the InputStream into String and manipulate as needed. There are many ways to do this. One of which can be found here: How can I convert InputStream data to String in Android SOAP Webservices
*Source: Android FTP connection Failed
If you get "425 Unable to build data connection: Connection timed out" error, then after connecting to ftp server, I would recommend you to set the local mode to passive mode by the following statement.
ftpClient.enterLocalPassiveMode();
I want to read the content of a remote directory using java.
The directory is on a machine running Ubuntu. Right clicking on the folder should give the share folder option and its installed samba client for windows sharing, but I don't have any machine running Windows.
I'm looking for a java api library to access the remote directory content?
User will only provide username, password, ip and folder name.
eg [//172.17.0.1/sharefolder/demo/]
Thanks.
For a Samba Share:
Even SAMBA shares in linux use the same protocol as windows shares.
So the post here can help: How can I mount a windows drive in Java?
Basically, you could mount the shared location as a network drive using "net use" command .
You could call this either through windows console, or through a java Process.
For a SFTP location:
If you don't have a problem with calling/using an external command you could use sshfs (either out of java or through Process) to mount the remote directory into a local folder.
See: http://numberformat.wordpress.com/2010/03/01/how-to-mount-a-remote-ssh-filesystem-using-sshfs/
If you want pure java on how to access SFTP,I read that a library called JSch can be used to access SFTP directly from java. See:
http://chrisjordan.ca/post/15052396308/java-sftp
http://jcraft.com/jsch/examples/Sftp.java.html
If it's another type please specify
You might find the The Java CIFS Client Library having the API you need - it is useful for both server and client.
Here is an example taken from their documentation to retrieve a file:
import jcifs.smb.*;
jcifs.Config.setProperty( "jcifs.netbios.wins", "192.168.1.220" );
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication("domain", "username", "password");
SmbFileInputStream in = new SmbFileInputStream("smb://host/c/My Documents/somefile.txt", auth);
byte[] b = new byte[8192];
int n;
while(( n = in.read( b )) > 0 ) {
System.out.write( b, 0, n );
}
For SFTP consider using JSCAPE's Secure FTP Factory. Documentation with code examples can be found here.
jsch-nio is a fully functional unix/linux java FileSystemProvider over ssh.