I am trying to get the list of files from remote FTP server. The ftpClient.listFiles() returned null and I had to set setUnparseableEntries to true to get the list of files. Even then the list of files do not have any information like name and only information it has is rawlisting and others are null. So I cannot do ftpFile.getName. Here is the code
public FTPFile[] process() throws Exception {
String message = null;
FTPFile[] files = null;
FTPClient ftpClient = new FTPClient();
FTPClientConfig config = new FTPClientConfig();
config.setServerTimeZoneId("America/Chicago");
config.setUnparseableEntries(true);
ftpClient.configure(config);
if ( !connectToServer() ) return null;
if ( !changeDirectory() ) {
disconnectFromServer();
return null;
}
files = getListofFiles();
disconnectFromServer();
return files;
}
private boolean connectToServer() {
boolean result = true;
String message = null, url = null;
// attempt to connect to the target server
try {
url = fo.getServerInfo().getConnectionURL();
LOGGER.debug("Connecting to: " + url);
ftpClient.connect(fo.getServerInfo().getHostName(),
fo.getServerInfo().getHostPort());
ftpClient.enterLocalPassiveMode();
} catch(SocketException e) {
result = false;
message = "Could not connect to server at " + url;
} catch(IOException e) {
result = false;
message = "Could not connect to server at " + url;
}
if ( !result ) return result;
// After connection attempt, you should check the reply code to verify success.
Integer replyCode = ftpClient.getReplyCode();
if ( !FTPReply.isPositiveCompletion(replyCode) ) {
message = "Reply Code - " + replyCode.toString() + " is negative.";
try {
ftpClient.disconnect();
} catch(Exception e) {
message = "Could not disconnect cleanly from server.";
LOGGER.error(message);
}
} else {
message = "Reply Code - " + replyCode.toString() + " is positive.";
}
Boolean logonOk = false;
try {
logonOk = ftpClient.login(fo.getServerInfo().getUserName(),
fo.getServerInfo().getUserPassword());
} catch(IOException e) {
message = "IOException during logon attempt.";
LOGGER.error(message);
}
if ( !logonOk ) {
result = false;
message = "Logon UNsuccessful.";
} else {
message = "Logon successful.";
LOGGER.error(message);
executionMessageLog.add(message);
}
if ( !result ) return result;
// attempt to log onto the target server
return result;
}
Following method is trying to get the list of files. I could see the file name using listNames and also listFiles shows list of files but name, modified date are empty and only has value in rawlisting in the format "04-01-20 11:31AM 8975 test.TXT". So how to get the name and modified date from the raw listing and why I could not get FTPFile name using getName
private FTPFile[] getListofFiles(){
String message = null;
FTPFile[] files = null;
try {
String[] filenames = ftpClient.listNames(fileListInfo.getFilePath());
files = ftpClient.listFiles(); /*Has only rawlisting and others are null*/
}
catch(IOException e) {
message = "IOException during getListofFiles attempt:";
LOGGER.error(message);
executionMessageLog.add(message);
message = e.getMessage();
LOGGER.error(message);
executionMessageLog.add(message);
}
return files;
}
04-01-20 11:31AM 8975 test.TXT
That's quite unusual format. So it's possible that Apache Commons Net library cannot parse it with the default configuration.
You might need to explicitly specify one of the available parsers. The available parsers are in src\main\java\org\apache\commons\net\ftp\parser. Or if there's no parser specifically compatible with your server, you might need to build your own (you can base it on ConfigurableFTPFileEntryParserImpl).
Though actually, for an ad-hoc solution, easier would be that you just parse the "rawlisting" you already have.
Related
I have a function, which generates a CSV file using CSV printer, emails and then deletes the file by calling another private function. The private function which sends the email is :
private void sendEmail(File fileToSend) {
List<String> emailTo = xyz#gmail.com
String emailFrom = abc#gmail.com
String host = host //cannot write the proper host name
try
{
mailUtil.sendMailWithAttachment(emailFrom, emailTo, host)
logger.debug("Email successfully sent");
} catch (Exception e) {
logger.error("Exception when sending mail ", e);
}
finally {
deleteFile(fileToSend);
}
}
private File saveInCsv(List<List <String> > valuesToStore) {
String fileName = "hello.csv";
File deletedIncidentCsv = new File(fileName);
try (BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter(deletedIncidentCsv));
CSVPrinter csvPrinter = new CSVPrinter(bufferedWriter, CSVFormat.EXCEL.withHeader(headers).withQuoteMode(QuoteMode.ALL));
)
{
for (List<String> strings : valuesToStore) {
csvPrinter.printRecord(strings);
}
csvPrinter.flush();
}
catch (Exception e)
{
logger.error("Error in writing to CSV file ", e);
}
return deletedIncidentCsv;
}
both these private functions are being called by public function , which I am testing:
public generateEmailFile(){
//does some work to retrieve dataToWriteInCSV
File csvFileGenerated = saveInCsv(dataToWriteInCsv);
sendEmail(csvFileGenerated);
}
In my unit test I want to check the contents of the file before its deleted. Is there anyway of doing it.
I am new in MQTT world. I have written a code to subscribe a topic and get message from topic and store it in database. Now my problem is how to put this code on server so that it will keep receiving message infinitely. I am trying to create a scheduler but in that case i am Getting Persistence Already in Use error from MQTT. I cannot change the clientId every time it connect. It is a fixed one in my case. Is there any way to get the persistence object which is already connected for a particular clientId?
Please help. Thanks and advance.
Please Find the code subscribe topic and messageArrived method of mqqt to get message from topic
public class AppTest {
private MqttHandler handler;
public void doApp() {
// Read properties from the conf file
Properties props = MqttUtil.readProperties("MyData/app.conf");
String org = props.getProperty("org");
String id = props.getProperty("appid");
String authmethod = props.getProperty("key");
String authtoken = props.getProperty("token");
// isSSL property
String sslStr = props.getProperty("isSSL");
boolean isSSL = false;
if (sslStr.equals("T")) {
isSSL = true;
}
// Format: a:<orgid>:<app-id>
String clientId = "a:" + org + ":" + id;
String serverHost = org + MqttUtil.SERVER_SUFFIX;
handler = new AppMqttHandler();
handler.connect(serverHost, clientId, authmethod, authtoken, isSSL);
// Subscribe Device Events
// iot-2/type/<type-id>/id/<device-id>/evt/<event-id>/fmt/<format-id>
handler.subscribe("iot-2/type/" + MqttUtil.DEFAULT_DEVICE_TYPE
+ "/id/+/evt/" + MqttUtil.DEFAULT_EVENT_ID + "/fmt/json", 0);
}
/**
* This class implements as the application MqttHandler
*
*/
private class AppMqttHandler extends MqttHandler {
// Pattern to check whether the events comes from a device for an event
Pattern pattern = Pattern.compile("iot-2/type/"
+ MqttUtil.DEFAULT_DEVICE_TYPE + "/id/(.+)/evt/"
+ MqttUtil.DEFAULT_EVENT_ID + "/fmt/json");
DatabaseHelper dbHelper = new DatabaseHelper();
/**
* Once a subscribed message is received
*/
#Override
public void messageArrived(String topic, MqttMessage mqttMessage)
throws Exception {
super.messageArrived(topic, mqttMessage);
Matcher matcher = pattern.matcher(topic);
if (matcher.matches()) {
String payload = new String(mqttMessage.getPayload());
// Parse the payload in Json Format
JSONObject contObj = new JSONObject(payload);
System.out
.println("jsonObject arrived in AppTest : " + contObj);
// Call method to insert data in database
dbHelper.insertIntoDB(contObj);
}
}
}
Code to connect to client
public void connect(String serverHost, String clientId, String authmethod,
String authtoken, boolean isSSL) {
// check if client is already connected
if (!isMqttConnected()) {
String connectionUri = null;
//tcp://<org-id>.messaging.internetofthings.ibmcloud.com:1883
//ssl://<org-id>.messaging.internetofthings.ibmcloud.com:8883
if (isSSL) {
connectionUri = "ssl://" + serverHost + ":" + DEFAULT_SSL_PORT;
} else {
connectionUri = "tcp://" + serverHost + ":" + DEFAULT_TCP_PORT;
}
if (client != null) {
try {
client.disconnect();
} catch (MqttException e) {
e.printStackTrace();
}
client = null;
}
try {
client = new MqttClient(connectionUri, clientId);
} catch (MqttException e) {
e.printStackTrace();
}
client.setCallback(this);
// create MqttConnectOptions and set the clean session flag
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(false);
options.setUserName(authmethod);
options.setPassword(authtoken.toCharArray());
//If SSL is used, do not forget to use TLSv1.2
if (isSSL) {
java.util.Properties sslClientProps = new java.util.Properties();
sslClientProps.setProperty("com.ibm.ssl.protocol", "TLSv1.2");
options.setSSLProperties(sslClientProps);
}
try {
// connect
client.connect(options);
System.out.println("Connected to " + connectionUri);
} catch (MqttException e) {
e.printStackTrace();
}
}
}
I'm working on Google Drive Integration using Java Drive Rest V2 API, I'm able to get most of the document/file metadata properties except path of the document/file.
I referred following StackOverflow questions as well:
How to get full file path from google drive using java
What's the right way to find files by “full path” in Google Drive API v2
In Both links solutions indicate that I have to create a separate method to achieve this requirement, It means there is no direct method provided by Drive Rest V2 API to get file path.
Please guide me and suggest your suggestion on this.
Thank you,
Arpit
Sharing my solution which will be helpful to others :)...
After several google searches and from Google Drive Documentation for Java Drive Rest V2 API, I got to know that there is no method to call/get the full file path.
So I created following two custom method to achieve my question solution:
"getFilePath(drive, file)" with String return type.
"getfoldersList(drive, parentReferencesList, folderList)" with List of String return type.
Below is the Code Snippet
public static void main(String[] args) throws IOException {
// Build a new authorized API client service.
Drive service = getDriveService();
try {
// Print the file path and name.
FileList result = service.files().list().execute();
List<File> files = result.getItems();
if (files == null || files.size() == 0) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
if (!(file.getMimeType().contains("folder"))) {
String filePath = null;
if (!(file.getParents().isEmpty())) {
filePath = getFilePath(service, file);
}
System.out.println("path: " + filePath);
System.out.println("name: " + file.getTitle());
System.out.println();
}
}
System.out.println("== END ==");
}
} catch (Exception e) {
System.out.println(e);
}
}
private static String getFilePath(Drive drive, File file) throws IOException {
String folderPath = "";
String fullFilePath = null;
List<ParentReference> parentReferencesList = file.getParents();
List<String> folderList = new ArrayList<String>();
List<String> finalFolderList = getfoldersList(drive, parentReferencesList, folderList);
Collections.reverse(finalFolderList);
for (String folder : finalFolderList) {
folderPath += "/" + folder;
}
fullFilePath = folderPath + "/" + file.getTitle();
return fullFilePath;
}
private static List<String> getfoldersList(Drive drive, List<ParentReference> parentReferencesList, List<String> folderList) throws IOException {
for (int i = 0; i < parentReferencesList.size(); i++) {
String id = parentReferencesList.get(i).getId();
File file = drive.files().get(id).execute();
folderList.add(file.getTitle());
if (!(file.getParents().isEmpty())) {
List<ParentReference> parentReferenceslist2 = file.getParents();
getfoldersList(drive, parentReferenceslist2, folderList);
}
}
return folderList;
}
--
Thanks,
Arpit Bora
Try using the Parents.get. A successful request returns a link to both parent and the file itself. Here's a sample response:
{
"kind": "drive#parentReference",
"id": "0Bw-w9jw2bwglbzlvMVh2cll2dmM",
"selfLink": "https://www.googleapis.com/drive/v2/files/1-ABCDEzyvp9NFmWz7h6ssgkTKHytO1Nq4SNIboDW8A/parents/0Bw-w9jw2bwglbzlvMVh2cll2dmM",
"parentLink": "https://www.googleapis.com/drive/v2/files/ABCDEjw2bwglbzlvMVh2cll2dmM",
"isRoot": false
}
Here's the JAVA implementation from the docs:
private static boolean isFileInFolder(Drive service, String folderId,
String fileId) throws IOException {
try {
service.parents().get(fileId, folderId).execute();
} catch (HttpResponseException e) {
if (e.getStatusCode() == 404) {
return false;
} else {
System.out.println("An error occured: " + e);
throw e;
}
} catch (IOException e) {
System.out.println("An error occured: " + e);
throw e;
}
return true;
}
I'm using org.apache.commons.net.ftp library to work with FTP servers. I use about 5 FTP servers and one of them doesn't work properly (for me). For uploading files I also use method ftpClient.setFileType(FTP.BINARY_FILE_TYPE), but on this server the size of the uploaded file is not the same as on my local HDD. It is usually MP3 files and when I try to play uploaded MP3 there is some buzz.
I suspects that FTP server has wrong configuration, but the company which takes care of this server would be very unlikely to do some changes.
One more very important note: when I use Total Commander or WinSCP to upload file, the uploaded file is correct with same size.
Here is my code:
public class FtpConnection {
String FTP_SERVER;
String FTP_USER;
String FTP_PASSWORD;
int FTP_PORT;
FTPClient ftpClient;
int attempts = 3;
long FILE_SIZE;
public FtpConnection() {
FTP_SERVER = StradaAd.FTP_SERVER;
FTP_USER = StradaAd.FTP_USER;
FTP_PASSWORD = StradaAd.FTP_PASSWORD;
FTP_PORT = StradaAd.FTP_PORT;
ftpClient = new FTPClient();
ftpClient.setCopyStreamListener(streamListener);
ftpClient.setBufferSize(1024 * 1024 * 1024);
ftpClient.setConnectTimeout(240000);
ftpClient.setDataTimeout(240000);
try {
ftpClient.connect(FTP_SERVER, FTP_PORT);
ftpClient.setControlKeepAliveTimeout(300);
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
int replyCode = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(replyCode)) {
StradaAd.errorArea.append("Operation failed. Server reply code: " + replyCode);
return;
}
boolean success = ftpClient.login(FTP_USER, FTP_PASSWORD);
if (!success) {
//unsuccess
}
} catch (IOException ex) {
//error
}
}
public boolean uploadFile(File source, String destination, String chain) {
try {
FILE_SIZE = source.length();
ftpClient.cwd("//" + chain + "/REKLAMY/");
for(int i=0;i<attempts;i++) {
ftpClient.storeFile(destination, new FileInputStream(source));
for(FTPFile f : ftpClient.listFiles()) {
if(f.getName().equals(destination) && f.getSize() == source.length())
return true;
}
}
return false;
} catch (IOException ex) {
Logger.getLogger(FtpConnection.class.getName()).log(Level.SEVERE, null, ex);
return false;
}
}
}
Do someone know the solution how to change this code or set this FTP server to have the same size uploaded file?
Thanks in advance for any help.
You call the FTPClient.setFileType method too early.
At least the ProFTPD or FileZilla FTP servers reset the default transfer mode after the authentication (FTPClient.login).
Move the .setFileType call after the .login.
SVN server is accessible over https. So I need to read a file that is located there. I followed the snippet from svnkit wiki (http://svn.svnkit.com/repos/svnkit/tags/1.3.5/doc/examples/src/org/tmatesoft/svn/examples/repository/DisplayFile.java), but my SVNKindNode is NONE and as a result no file is read. Nevertheless there's no exceptions during connection. So I can assume that I do connect correctly to SVN server, but then something goes wrong.
Here is the code:
public class SVNRepoConnector {
private String username = "user";
private String password = "pwd";
private String baseUrl = "https://mysvnserver.com/svn/project/trunk";
private String filePath = "/myproject/src/main/webapp/file.html";
public void downloadSchema() {
DAVRepositoryFactory.setup();
SVNRepository repository = null;
try {
repository = SVNRepositoryFactory.create(SVNURL.parseURIEncoded(baseUrl));
ISVNAuthenticationManager authManager = SVNWCUtil.createDefaultAuthenticationManager(username, password);
repository.setAuthenticationManager(authManager);
SVNNodeKind nodeKind = repository.checkPath(filePath, -1);
if(nodeKind == SVNNodeKind.NONE) {
System.err.println("There is file at: " + baseUrl + filePath);
System.exit(1);
} else if (nodeKind == SVNNodeKind.DIR) {
System.err.println("The entry at " + baseUrl + filePath + " is a directory while a file was expected.");
System.exit(1);
}
SVNProperties properties = new SVNProperties();
ByteArrayOutputStream out = new ByteArrayOutputStream();
repository.getFile(filePath, -1, properties, out);
System.out.println("Content:\n");
try {
out.writeTo(System.out);
} catch (IOException e) {
e.printStackTrace();
}
} catch (SVNException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
SVNRepoConnector connector = new SVNRepoConnector();
connector.downloadSchema();
}
}
I receive "There is file at..." due to SVNNodeKind equals NONE. I cannot understand what is wrong here. How to read file from SVN over https?
Btw, my svnkit is 1.8.5.
Specify a relative path (unless baseUrl is the repository root):
private String filePath = "myproject/src/main/webapp/file.html";
instead of
private String filePath = "/myproject/src/main/webapp/file.html";
I found the solution after thorough debugging of the sources.
In short, the problem is in the second argument of repository.checkPath(filePath, -1); and repository.getFile(filePath, -1, properties, out);. filePath must be file name and path to it must be in the baseUrl field. After these changes everything started working correctly.
Regarding the snippet, in case of www/license.html, one should pass 1 as a second arg.