I am trying to access a file via a URI using the FTP protocol. For obvious security reasons I had to make some changes but this is where the problems seem to be coming from.
My URI is as follows:
ftp://user:pasword#host.net/u/Bigpathname/XYZ/ABC/BigPathname/bigpathname/xyz/abc/MY_LOG.LOG
And I see this exception:
sun.net.ftp.FtpProtocolException: CWD Bigpathname:501 A qualifier in "Bigpathname" is more than 8 characters
This is really confusing as I can access the file from a Windows 7 command line with the CD command just fine. Both one directory at a time and as a full path.
I found one article mentioning that MVS file names must be 8 or fewer characters but this does not explain how I can get to these same files from my command line! They do exist there is data there that I can download manual but I can not get there via a URI in Java.
PS I use .toURL().openStream() to get files on my local machine just fine, it only fails when I try to get them from my server.
EDIT October 1st
I am able to access files on the MVS host using FileZilla and the basic FTP client from the Windows 7 command line - but I still cannot get them from a URI/URL. I downloaded a very basic Java built FTP client and tried accessing the same file in my program from there and the path works but because my file name has a dot in it "MY_LOG.LOG" I am getting File does not exist 501 Invalid data set name "MY_LOG.LOG". Use MVS Dsname conventions. I am utterly perplexed by this...
EDIT Ocotober 1st afternoon :)
OK I finally got it to work with a FTP client in my Java code - but I still want to use the URL class as I have logs on both local and remote machines. Is there a way to encode a URL string so that it can retrieve a file from a remote machine with the FTP protocol? I am not sure how it works in the Java URL class but in the FTP client I had to use the CWD and then the RETR command.
If I can do this then I have one solution for getting all my logs, otherwise I will have to detect if it is a file or ftp URL and then behave differently. Not the end of the world but not what I want...
The code that tries to get the file with just a URL is as follows: (sysc is a valid host)
void testFTP()
{
String ftp = "ftp://user:pword#sysc/u/Xxxxxxxxxx/ICS/YT7/XxxxxXxxxxxxx/xxxxxxxxx/logs/xxxxxxxx/XX_YT.LOG";
try
{
URI uri = new URI(ftp);
URL ftpFile = uri.toURL();
BufferedReader in = new BufferedReader(new InputStreamReader(ftpFile.openStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
catch(Exception e)
{
e.printStackTrace();
}
}
In this case I think the problem is also Server Related, It all works fine for me with Filezilla Server except when the filename length(including directories) exceeds 255 chars but if you want to use the URL class with another FTP you must override or implement your own URLStreamHandlerFactory.
URL.setURLStreamHandlerFactory(...);
I haven't found any for my favorite java FTP Client witch is Apache one so I have developed one but may need a few touch ups.
package net.custom.streamhandler.apacheftp;
import java.io.IOException;
import java.io.InputStream;
import java.net.SocketException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLStreamHandler;
import java.net.URLStreamHandlerFactory;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
public class ApacheURLStreamHandlerFactory implements URLStreamHandlerFactory {
public URLStreamHandler createURLStreamHandler(String protocol) {
//this will only override the chosen protocol
if ( protocol.equalsIgnoreCase("ftp") )
return new CustomHandler();
else
return null;
}
}
class CustomHandler extends URLStreamHandler {
protected URLConnection openConnection(URL url)
throws IOException {
return new CustomURLConnection(url);
}
}
class CustomURLConnection extends URLConnection {
int reply;
FTPClient ftp = new FTPClient();
InputStream in;
static int defaultPort = 21;
static String defaultPath = "/";
CustomURLConnection ( URL url)
throws IOException {
super( url );
}
synchronized public void connect() throws IOException {
try {
int port;
if ((port = url.getPort()) == -1 )
port = defaultPort;
ftp.connect(url.getHost(), port);
String login = "anonymous";
String password = "";
if(url.getAuthority().indexOf(':')>-1 &&
url.getAuthority().indexOf('#')>-1){
String []auxArray = url.getAuthority().replaceAll("#", ":").split(":");
login = auxArray[0];
password = auxArray[1];
}
ftp.login(login, password);
reply = ftp.getReplyCode();
if (FTPReply.isPositiveCompletion(reply)) {
System.out.println("Connected Apache Success");
} else {
System.out.println("Connection Apache Failed");
ftp.disconnect();
}
in = ftp.retrieveFileStream(url.getFile());
} catch (SocketException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
connected = true;
}
synchronized public InputStream getInputStream()
throws IOException {
if (!connected)
connect();
return ( in );
}
}
*Keep in mind that you can implement new ways to handle different protocols for the java.net.URL this way.
Your code...
...
{
String ftp = "ftp://user:pword#sysc/u/Xxxxxxxxxx/ICS/YT7/XxxxxXxxxxxxx/xxxxxxxxx/logs/xxxxxxxx/XX_YT.LOG";
try
{
URL.setURLStreamHandlerFactory(new ApacheURLStreamHandlerFactory());
...
G'Bye
**(To err is human, to forgive is divine)
Try using the short name for the path. Something like /U/BIGPAT~1/XYZ/ABC/BIGPAT~1/BIGPAT~1/XYZ/ABC/MY_LOG.LOG
You can find the short name for any directory longer than 8 characters with dir /x.
FTP clients are notoriously difficult to write given the variation of (and bugs in) server implementations.
I'm betting that MVS is not completely supported by sun.net.ftp.FtpClient, which is the class used under the hood when you call URL.openStream on an FTP URL.
The Apache Commons Net library should support MVS, but it sounds like you already found a working client.
Have you considered using an RMI for transporting the files that way you can give a direct path to the file as a parameter without the use of ftp then have the file sent back in a byte array.
Related
I want to attach multiple files to issue. I'm able to create issue successfully however i am facing problem in attaching documents after creating issue. I have referred to this link SOLVED: attach a file using REST from scriptrunner
I am getting 404 error even though issue exists and also user has all the permissions.
File fileToUpload = new File("D:\\dummy.txt");
InputStream in = null;
try {
in = new FileInputStream(fileToUpload);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
HttpResponse < String > response3 = Unirest
.post("https://.../rest/api/2/issue/test-85/attachments")
.basicAuth(username, password).field("file", in , "dummy.txt")
.asString();
System.out.println(response3.getStatus());
here test-85 is a issueKey value.
And i am using open-unirest-java-3.3.06.jar. Is the way i am attaching documents is correct?
I am not sure how open-unirest manages its fields, maybe it tries to put them as json field, rather than post content.
I've been using Rcarz's Jira client. It's a little bit outdated but it still works.
Maybe looking at its code will help you, or you can just use it directly.
The Issue class:
public JSON addAttachment(File file) throws JiraException {
try {
return restclient.post(getRestUri(key) + "/attachments", file);
} catch (Exception ex) {
throw new JiraException("Failed add attachment to issue " + key, ex);
}
}
And in RestClient class:
import org.apache.http.client.methods.HttpEntityEnclosingRequestBase;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.FileBody;
public JSON post(String path, File file) throws RestException, IOException, URISyntaxException {
return request(new HttpPost(buildURI(path)), file);
}
private JSON request(HttpEntityEnclosingRequestBase req, File file) throws RestException, IOException {
if (file != null) {
File fileUpload = file;
req.setHeader("X-Atlassian-Token", "nocheck");
MultipartEntity ent = new MultipartEntity();
ent.addPart("file", new FileBody(fileUpload));
req.setEntity(ent);
}
return request(req);
}
So I'm not sure why you're getting a 404, Jira is sometime fuzzy and not really clear about its error, try printing the full error, or checking Jira's log if you can. Maybe it's just the "X-Atlassian-Token", "nocheck", try adding it.
I have an akka http service. I simply return the api documentation for a get request. The documentation is in html file.
It all works fine when run within the IDE. When I package it as a jar I get error 'resource not found'. I am not sure why it can not read the html file when hosted in a jar and works fine when in IDE.
Here is the code for the route.
private Route topLevelRoute() {
return pathEndOrSingleSlash(() -> getFromResource("asciidoc/html/api.html"));
}
The files are located in resource path.
I have got this working now.
I am doing this.
private Route topLevelRoute() {
try {
InputStreamReader inputStreamReader = new InputStreamReader(getClass().getResourceAsStream("/asciidoc/html/api.html"));
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
//Get the stream input into string builder
reader.lines().forEach(s -> strBuild.append(s));
inputStreamReader.close();
bufferedReader.close();
//pass the string builder as string with contenttype set to html
complete(HttpEntities.create(ContentTypes.TEXT_HTML_UTF8, strBuild.toString()))
} catch (Exception ex) {
//Catch any exception here
}
}
I am developing a code for xcel generation and download using apache poi. LocalHost server and app server is jboss. When i run the code on localhost, a temp folder is generated in jboss's deployment folder and in that the xcel is generated and then downloaded through frontend. I am using java spring angularjs and html. This runs fine on localhost but after deploying on app server the xcel is not downloaded and it gives 500:internal server error.
angularjs controller code:
$scope.generateExcel=function(sDate,eDate,doc,search)
{
console.log("hello");
var sDate = document.getElementById('sD').value
var eDate = document.getElementById('eD').value
$scope.obj.sDate = sDate;
$scope.obj.eDate = eDate;
$scope.obj.iou = doc;
$scope.obj.du = search;
console.log($scope.obj);
$http.post('abc/generateExcel',$scope.obj).then(function()
{
//console.log(path);
$window.location.href="/ProjectName/file_name.xls";
})
.error(function()
{
console.log("Error!!");
});
};
java code:
//Method
public HttpServletResponse generateExcel ( HttpServletRequest request , HttpServletResponse response, String sD, String eD, String doc, String search)
{
//EXCEL GENERATION HERE
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment;filename=filename.xls");
//Path Specification
String path = request.getRealPath("/file_name.xls");
//System.out.println("Here...");
System.out.println(path);
FileOutputStream fileOut2 = new FileOutputStream(path);
workbook.write(fileOut2);
/*returning response*/
}
It's difficult to answer this unless one knows what's the error you are getting on the server side. Put your server code in a try-catch block. Rerun the code, and check the server logs. Paste them here.
try{
String path = request.getRealPath("/file_name.xls");
//System.out.println("Here...");
System.out.println(path);
FileOutputStream fileOut2 = new FileOutputStream(path);
workbook.write(fileOut2);
} catch(Exception e){
e.printStackTrace(); // this should print some error in server logs
}
I stole this code to test about emailing using java. Javamail is required, obviously. For some reason, I can't get javax.mail to implement. I downloaded the most recent javamail and put them in the jdk and jre lib folders, yet nothing changes. Please and thank you!
//A class which uses this file to send an email :
import java.util.*;
import java.io.*;
import javax.mail.*;
import javax.mail.internet.*;
/**
* Simple demonstration of using the javax.mail API.
*
* Run from the command line. Please edit the implementation
* to use correct email addresses and host name.
*/
public final class Emailer {
public static void main( String... aArguments ){
Emailer emailer = new Emailer();
//the domains of these email addresses should be valid,
//or the example will fail:
emailer.sendEmail(
"sean_chili#yahoo.com", "clevelanm#sou.edu",
"Testing 1-2-3", "blah blah blah"
);
}
/**
* Send a single email.
*/
public void sendEmail(
String aFromEmailAddr, String aToEmailAddr,
String aSubject, String aBody
){
//Here, no Authenticator argument is used (it is null).
//Authenticators are used to prompt the user for user
//name and password.
Session session = Session.getDefaultInstance( fMailServerConfig, null );
MimeMessage message = new MimeMessage( session );
try {
//the "from" address may be set in code, or set in the
//config file under "mail.from" ; here, the latter style is used
//message.setFrom( new InternetAddress(aFromEmailAddr) );
message.addRecipient(
Message.RecipientType.TO, new InternetAddress(aToEmailAddr)
);
message.setSubject( aSubject );
message.setText( aBody );
Transport.send( message );
}
catch (MessagingException ex){
System.err.println("Cannot send email. " + ex);
}
}
/**
* Allows the config to be refreshed at runtime, instead of
* requiring a restart.
*/
public static void refreshConfig() {
fMailServerConfig.clear();
fetchConfig();
}
// PRIVATE //
private static Properties fMailServerConfig = new Properties();
static {
fetchConfig();
}
/**
* Open a specific text file containing mail server
* parameters, and populate a corresponding Properties object.
*/
private static void fetchConfig() {
InputStream input = null;
try {
//If possible, one should try to avoid hard-coding a path in this
//manner; in a web application, one should place such a file in
//WEB-INF, and access it using ServletContext.getResourceAsStream.
//Another alternative is Class.getResourceAsStream.
//This file contains the javax.mail config properties mentioned above.
input = new FileInputStream( "C:\\Temp\\MyMailServer.txt" );
fMailServerConfig.load( input );
}
catch ( IOException ex ){
System.err.println("Cannot open and load mail server properties file.");
}
finally {
try {
if ( input != null ) input.close();
}
catch ( IOException ex ){
System.err.println( "Cannot close mail server properties file." );
}
}
}
}
Just for completeness, here's the answer.
Your Eclipse is telling you
<Some Class> cannot be resolved to a type
This is usually an indication that your classpath is not correct. You said
I downloaded the most recent javamail and put them in the jdk and jre
lib folders, yet nothing changes
Don't do this. Take the javamail.jar and use it on your application Build Path. To do so, drag and drop the jar into your project, right-click it and select Build Path > Add to build path.
I am using org.apache.commons.net.ftp.FTPClient in one of my applications to work with a FTP server. I am able to connect, login, pwd and cwd. However, when I try to list the files it doesn't return the list of files in that directory, where I know for sure that there are files. I am using the method FTPFile[] listFiles(), it returns an empty array of FTPFile.
Please find below the code snippet where I am trying this:
String hostname = properties.getProperty("FTP_SERVER");
String user = properties.getProperty("FTP_USER");
String passwd = properties.getProperty("FTP_PASSWD");
FTPClient client = new FTPClient();
client.connect(hostname);
client.login(user, passwd);
String reply = client.getStatus();
System.out.println(reply);
client.enterRemotePassiveMode();
client.changeWorkingDirectory("/uploads");
FTPFile[] files = client.listFiles();
System.out.println(files.length);
for (FTPFile file : files) {
System.out.println(file.getName());
}
String[] fileNames = client.listNames();
if (fileNames != null) {
for (String file : fileNames) {
System.out.println(file);
}
}
client.disconnect();
This seems like the same issue I had (and solved), see this answer:
Apache Commons Net FTPClient and listFiles()
After I set the mode as PASV it is working fine now!
Thanks for all your efforts and suggestions!
I added client.enterLocalPassiveMode() and it works:
client.connect("xxx.com");
boolean login = client.login("xxx", "xxx");
client.enterLocalPassiveMode();
Just a silly suggestion... can you do a listing on the /uploads folder using a normal FTP client. I ask this because some FTP servers are setup to not display the listing of an upload folder.
First, make sure the listing works in other programs. If so, one possibility is that the file listing isn't being parsed correctly. You can try explicitly specifying the parser to use with initiateListParsing.
I had to same problem and it turned out to be that it couldn't parse what the server was returning for a file listing. I this line after connecting to the ftp server ftpClient.setParserFactory(new MyFTPFileEntryParserFactory());
public class MyFTPFileEntryParserFactory implements FTPFileEntryParserFactory {
private final static FTPFileEntryParser parser = new UnixFTPEntryParser() {
#Override public FTPFile parseFTPEntry(String entry) {
FTPFile ftpFile = new FTPFile();
ftpFile.setTimestamp(getCalendar(entry));
ftpFile.setSize(get(entry));
ftpFile.setName(getName(entry));
return ftpFile;
}
};
#Override public FTPFileEntryParser createFileEntryParser(FTPClientConfig config) throws ParserInitializationException {
return parser;
}
#Override public FTPFileEntryParser createFileEntryParser(String key) throws ParserInitializationException {
return parser;
}
}
In my case, on top of applying enterLocalPassiveMode and indicating correct operation system, I also need to set UnparseableEntries to true to make the listFile method work.
FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
conf.setUnparseableEntries(true);
f.configure(conf);
boolean isLoginSuccess = client.login(username, password);