Develop Reference

Java Current Directory Returns `null`

I'm trying to use the printWorkingDirectory() from Apache Commons FTP but it's only returning null. I can't navigate directories, list files, etc.
Log in pass all is success but how ever I try I can not change current directory.
I use this following code:
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
public class FTPDownloadFileDemo {
public static void main(String[] args) {
String server = "FTP server Address";
int port = portNo;
String user = "User Name";
String pass = "Pasword";
FTPClient ftpClient = new FTPClient();
String dir = "stocks/";
try {
ftpClient.connect(server, port);
ftpClient.login(user, pass);
System.out.println( ftpClient.printWorkingDirectory());//Always null
//change current directory
ftpClient.changeWorkingDirectory(dir);
boolean success = ftpClient.changeWorkingDirectory(dir);
// showServerReply(ftpClient);
if (success)// never success
System.out.println("Successfully changed working directory.");
System.out.println(ftpClient.printWorkingDirectory());// Always null
} catch (IOException ex) {
System.out.println("Error: " + ex.getMessage());
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}

This is rather old question that deserves an answer. This issue is likely a result of using FTPClient when secure connection is required. You may have to switch to FTPSClient if that is, indeed, the case. Further, output the response from the server with the following code snippet to troubleshoot the issue if secure client doesn't solve the it:
ftpClient.addProtocolCommandListener(
new PrintCommandListener(
new PrintWriter(new OutputStreamWriter(System.out, "UTF-8")), true));
Also, a server can reject your login attempt if your IP address is not white listed. So, being able to see the logs is imperative. The reason you see null when printing current working directory is because you are not logged in. Login method will not throw an exception but rather return a boolean value indicating if the operation succeeded. You are checking for success when changing a directory but not doing so when logging in.
boolean success = ftpClient.login(user, pass);

I faced the same, but I came across with a simple step.
Just added this.
boolean success = ftpClient.changeWorkingDirectory(dir);
ftpClient.printWorkingDirectory(); //add this line after changing the working directory
System.out.println(ftpClient.printWorkingDirectory()); //wont be getting null
Here I have the code and the console output
FTPClient.changeWorkingDirectory - Unknown parser type: "/Path" is current directory
I know I replied too soon ;-P, but I saw this post recently. Hope this helps to future searchers ;-)

NoSuchMethodError showing for Jira when Java running as Web application, But gets output when running in main

I am creating a Java web application. I added Jira API jars for connecting to Jira to get Issues created in Jira.
And this is my Java program to get the issues
import java.net.URI;
import java.net.URISyntaxException;
import com.atlassian.jira.rest.client.JiraRestClient;
import com.atlassian.jira.rest.client.NullProgressMonitor;
import com.atlassian.jira.rest.client.domain.Issue;
import com.atlassian.jira.rest.client.internal.jersey.JerseyJiraRestClientFactory;
public class AppJIRADataCollector {
final static JerseyJiraRestClientFactory factory = new JerseyJiraRestClientFactory();
public static void main(String args[])
{
AppJIRADataCollector appJira=new AppJIRADataCollector();
appJira.getIssues("http://localhost:8080/", "AP-1", "admin", "admin");
}
public void getIssues(String issueURI,String issueKey,String username, String password) {
try {
final URI jiraServerUri = new URI(issueURI);
final JiraRestClient restClient = factory.createWithBasicHttpAuthentication(jiraServerUri,username, password);
System.out.println(restClient);
final NullProgressMonitor pm = new NullProgressMonitor();
final Issue issue = restClient.getIssueClient()
.getIssue(issueKey, pm);
System.out.println(issue);
} catch (URISyntaxException e) {
System.out.println("URI not found");
e.printStackTrace();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
When I run this program, I am getting the output correctly as it prints the issues.
But when I run this as web application, and calling the getIssues() method from servlet, tomcat showing NoSuchMethodError
Here is the stack trace
java.lang.NoSuchMethodError: com.google.common.collect.Iterables.contains(Ljava/lang/Iterable;Ljava/lang/Object;)Z
com.atlassian.jira.rest.client.internal.json.IssueJsonParser.parse(IssueJsonParser.java:216)
com.atlassian.jira.rest.client.internal.json.IssueJsonParser.parse(IssueJsonParser.java:59)
com.atlassian.jira.rest.client.internal.jersey.AbstractJerseyRestClient$1.call(AbstractJerseyRestClient.java:85)
com.atlassian.jira.rest.client.internal.jersey.AbstractJerseyRestClient.invoke(AbstractJerseyRestClient.java:54)
com.atlassian.jira.rest.client.internal.jersey.AbstractJerseyRestClient.getAndParse(AbstractJerseyRestClient.java:80)
com.atlassian.jira.rest.client.internal.jersey.JerseyIssueRestClient.getIssue(JerseyIssueRestClient.java:131)
com.atlassian.jira.rest.client.internal.jersey.JerseyIssueRestClient.getIssue(JerseyIssueRestClient.java:123)
com.domain.jirademo.adapters.AppJIRADataCollector.getIssues(AppJIRADataCollector.java:22)
com.domain.jirademo.controllers.ProjectDefectsServlet.doPost(ProjectDefectsServlet.java:50)
javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
Why this is happening? Any problem with JARs? I am using maven. If want I will post the pom also.
EDIT
I added the main method later only. So in the stack trace, line 22 refers to
final Issue issue = restClient.getIssueClient()
.getIssue(issueKey, pm);

NoSuchMethodError it's caused by a version mismatch, you should check your version of guava library.

Using NTLM authentication in Java applications

I want to use Windows NTLM authentication in my Java application to authenticate intranet users transparently. The users should not notice any authentication if using their browsers (single sign-on).
I've found a few libs with NTLM support, but don't know which one to use:
http://spnego.sourceforge.net/
http://sourceforge.net/projects/ntlmv2auth/
http://jcifs.samba.org/
http://www.ioplex.com/jespa.html
http://www.luigidragone.com/software/ntlm-authentication-in-java/
Any suggestions where to start?

Out of the above list, only ntlmv2-auth and Jespa support NTLMv2. Jespa is workable but commercial. ntlmv2-auth I haven't tried but it's based on the code from Liferay, which I've seen working before.
'ntlm-authentication-in-java' is only NTLMv1, which is old, insecure, and works in a dwindling number of environments as people upgrade to newer Windows versions. JCIFS used to have an NTLMv1 HTTP auth filter, but it was removed in later versions, as the way it was implemented amounts to a man-in-the-middle attack on the insecure protocol. (The same appears to be true of 'ntlm-authentication-in-java'.)
The 'spnego' project is Kerberos not NTLM. If you want to replicate full IWA as IIS does it, you'd need to support both NTLMv2 and Kerberos ('NTLM' auth, 'Negotiate' auth, NTLMSSP-in-SPNego auth and NTLM-masquerading-as-Negotiate auth).

Luigi Dragone's script is really old and seems to always fail.
HttpURLConnection can work with NTLM if you add library jcifs, this example works with latest jcifs-1.3.18 :
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;
import org.apache.http.impl.auth.NTLMEngineException;
public class TestNTLMConnection {
public static void main(String[] args) throws UnknownHostException, IOException, NTLMEngineException {
// Method 1 : authentication in URL
jcifs.Config.registerSmbURLHandler();
URL urlRequest = new URL("http://domain%5Cuser:pass#127.0.0.1/");
// or Method 2 : authentication via System.setProperty()
// System.setProperty("http.auth.ntlm.domain", "domain");
// System.setProperty("jcifs.smb.client.domain", "domain");
// System.setProperty("jcifs.smb.client.username", "user");
// System.setProperty("jcifs.smb.client.password", "pass");
// Not verified // System.setProperty("jcifs.netbios.hostname", "host");
// System.setProperty("java.protocol.handler.pkgs", "jcifs");
// URL urlRequest = new URL("http://127.0.0.1:8180/simulate_get.php");
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
StringBuilder response = new StringBuilder();
try {
InputStream stream = conn.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(stream));
String str = "";
while ((str = in.readLine()) != null) {
response.append(str);
}
in.close();
System.out.println(response);
} catch(IOException err) {
System.out.println(err);
} finally {
Map<String, String> msgResponse = new HashMap<String, String>();
for (int i = 0;; i++) {
String headerName = conn.getHeaderFieldKey(i);
String headerValue = conn.getHeaderField(i);
if (headerName == null && headerValue == null) {
break;
}
msgResponse.put(headerName == null ? "Method" : headerName, headerValue);
}
System.out.println(msgResponse);
}
}
}
And if you are curious about the content of each handshake, you can find another example using jcifs and Socket on this thread.

Had to recently implement this at work hence here is updated solution with Spring's RestTemplate:
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClients;
import org.springframework.http.HttpEntity;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import java.io.IOException;
public class Runner {
public static void main(String[] args) {
var credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new NTCredentials("username", "password", "", "someDomain"));
try (var client = HttpClients.custom()
.setDefaultCredentialsProvider(credentialsProvider)
.build();) {
var requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(client);
RestTemplate restTemplate = new RestTemplate(requestFactory);
ResponseEntity<String> stringResponseEntity = restTemplate.postForEntity("url", new HttpEntity<>("yourDtoObject"), String.class);
} catch (IOException e) {
e.printStackTrace();
}
}
}
dependencies needed are: spring-web and org.apache.httpcomponents
ps: it is important to enter username without domain otherwise it doesn't work. As in if your domain is companyName/username often people just enter that whole thing as username and what you should do is enter them separately where domain="companyName" and username="username"

Ref: https://jcifs.samba.org/src/docs/faq.html#ntlmv2
Q: Does jCIFS support NTLMv2?
A: Yes. As of 1.3.0, JCIFS fully supports NTLMv2 and uses it by default.
Note: The NTLM HTTP SSO Filter that used to be included with JCIFS cannot support NTLMv2.

Relatively from the list you gave,I would go with JCIFS.
The library is mature , and their documentation is good.
To top it off they had fairly regular releases , and the last one being Nov 2011.
Personal Experience : it was fairly easy to get started when compared to others i have tried (spnego and ntmv2auth)

java flickr and flickrj download user pictures

Hi I am new to flickrj library.
Have foundational java knowledge though.
The project that I am working on requires me to authenticate into flickr and then download geo-tagged images into a folder in local hard drive. The program will be Desktop application program.
I am approaching the program by breaking down into 3 steps.
1.Proper authentication to be completed.(which i have succeeded)
2.Try to download all the photos that user has when authenticated.
3.Try to alter the code a little so that it will only download geo-tagged images.
My problems is on step 2. I cant download logged-in user images let alone geo-tagged ones.
I am trying the code provided by Daniel Cukier here
But I am running into problem.
My netbeans simply strike off at the line 77 on .getOriginalAsStream() part, with the error "java.lang.RuntimeException: Uncompilable source code - Erroneous sym type: java.io.ByteArrayOutputStream.write"
From my understanding netbeans striking off a line means , it is depreciated but shouldnt it still work? What is holding this whole problem back?
I have tried researching and basically I have to admit , it is beyond my capability to trouble shoot. If anyone has any idea on what i am doing wrong , I would be so grateful.
Ps: I am not looking to be spoon fed but please answer me in idiot-friendly way as I am still a student and my java isn't the greatest.
This code is what I have so far.
import com.aetrion.flickr.*;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.Properties;
import javax.xml.parsers.ParserConfigurationException;
import org.xml.sax.SAXException;
import com.aetrion.flickr.auth.Auth;
import com.aetrion.flickr.auth.AuthInterface;
import com.aetrion.flickr.auth.Permission;
import com.aetrion.flickr.photos.Photo;
import com.aetrion.flickr.photos.PhotoList;
import com.aetrion.flickr.photos.PhotosInterface;
import com.aetrion.flickr.util.IOUtilities;
import java.io.*;
import java.util.Iterator;
import org.apache.commons.io.FileUtils;
public class authenticate {
Flickr f;
RequestContext requestContext;
String frob = "";
String token = "";
Properties properties = null;
public authenticate() throws ParserConfigurationException, IOException, SAXException {
InputStream in = null;
try {
in = getClass().getResourceAsStream("/setup.properties");
properties = new Properties();
properties.load(in);
} finally {
IOUtilities.close(in);
}
f = new Flickr(
properties.getProperty("apiKey"),
properties.getProperty("secret"),
new REST()
);
Flickr.debugStream = false;
requestContext = RequestContext.getRequestContext();
AuthInterface authInterface = f.getAuthInterface();
try {
frob = authInterface.getFrob();
} catch (FlickrException e) {
e.printStackTrace();
}
System.out.println("frob: " + frob);
URL url = authInterface.buildAuthenticationUrl(Permission.DELETE, frob);
System.out.println("Press return after you granted access at this URL:");
System.out.println(url.toExternalForm());
BufferedReader infile =
new BufferedReader ( new InputStreamReader (System.in) );
String line = infile.readLine();
try {
Auth auth = authInterface.getToken(frob);
System.out.println("Authentication success");
// This token can be used until the user revokes it.
System.out.println("Token: " + auth.getToken());
System.out.println("nsid: " + auth.getUser().getId());
System.out.println("Realname: " + auth.getUser().getRealName());
System.out.println("Username: " + auth.getUser().getUsername());
System.out.println("Permission: " + auth.getPermission().getType());
PhotoList list = f.getPhotosetsInterface().getPhotos("72157629794698308", 100, 1);
for (Iterator iterator = list.iterator(); iterator.hasNext();) {
Photo photo = (Photo) iterator.next();
File file = new File("/tmp/" + photo.getId() + ".jpg");
ByteArrayOutputStream b = new ByteArrayOutputStream();
b.write(photo.getOriginalAsStream());
FileUtils.writeByteArrayToFile(file, b.toByteArray());
}
} catch (FlickrException e) {
System.out.println("Authentication failed");
e.printStackTrace();
}
}
public static void main(String[] args) {
try {
authenticate t = new authenticate();
} catch(Exception e) {
e.printStackTrace();
}
System.exit(0);
}
}

You are correct in your interpretation of the strikeout that getOriginalAsStream() is deprecated. It looks like you might want to rework your code to use PhotosInterface.getImageAsStream(), passing the ORIGINAL size as one of the arguments.
To adjust NetBeans' behavior with respect to deprecated methods, you can follow the link recommended by #AljoshaBre as well as this one.

If you want download all your photos from Flickr, this is possible if you have a mac computer.
Download Aperture program on Apple Store and install it.
After to install, open the Aperture.
Go on preferences.
Click on 'Accounts' tab.
Click on plus sign (+) on bottom left to add a photo service.
Add the Flicker option.
Follow the login and authorization instructions.
Done! All your photos will be synchronized in you aperture library locate on ~/images/
I hope I have helped.

Using Java to pull data from a webpage?

I'm attempting to make my first program in Java. The goal is to write a program that browses to a website and downloads a file for me. However, I don't know how to use Java to interact with the internet. Can anyone tell me what topics to look up/read about or recommend some good resources?

The simplest solution (without depending on any third-party library or platform) is to create a URL instance pointing to the web page / link you want to download, and read the content using streams.
For example:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class DownloadPage {
public static void main(String[] args) throws IOException {
// Make a URL to the web page
URL url = new URL("http://stackoverflow.com/questions/6159118/using-java-to-pull-data-from-a-webpage");
// Get the input stream through URL Connection
URLConnection con = url.openConnection();
InputStream is = con.getInputStream();
// Once you have the Input Stream, it's just plain old Java IO stuff.
// For this case, since you are interested in getting plain-text web page
// I'll use a reader and output the text content to System.out.
// For binary content, it's better to directly read the bytes from stream and write
// to the target file.
try(BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
String line = null;
// read each line and write to System.out
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
}
}
Hope this helps.

The Basics
Look at these to build a solution more or less from scratch:
Start from the basics: The Java Tutorial's chapter on Networking, including Working With URLs
Make things easier for yourself: Apache HttpComponents (including HttpClient)
The Easily Glued-Up and Stitched-Up Stuff
You always have the option of calling external tools from Java using the exec() and similar methods. For instance, you could use wget, or cURL.
The Hardcore Stuff
Then if you want to go into more fully-fledged stuff, thankfully the need for automated web-testing as given us very practical tools for this. Look at:
HtmlUnit (powerful and simple)
Selenium, Selenium-RC
WebDriver/Selenium2 (still in the works)
JBehave with JBehave Web
Some other libs are purposefully written with web-scraping in mind:
JSoup
Jaunt
Some Workarounds
Java is a language, but also a platform, with many other languages running on it. Some of which integrate great syntactic sugar or libraries to easily build scrapers.
Check out:
Groovy (and its XmlSlurper)
or Scala (with great XML support as presented here and here)
If you know of a great library for Ruby (JRuby, with an article on scraping with JRuby and HtmlUnit) or Python (Jython) or you prefer these languages, then give their JVM ports a chance.
Some Supplements
Some other similar questions:
Scrape data from HTML using Java
Options for HTML Scraping

Here's my solution using URL and try with resources phrase to catch the exceptions.
/**
* Created by mona on 5/27/16.
*/
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
public class ReadFromWeb {
public static void readFromWeb(String webURL) throws IOException {
URL url = new URL(webURL);
InputStream is = url.openStream();
try( BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
catch (MalformedURLException e) {
e.printStackTrace();
throw new MalformedURLException("URL is malformed!!");
}
catch (IOException e) {
e.printStackTrace();
throw new IOException();
}
}
public static void main(String[] args) throws IOException {
String url = "https://madison.craigslist.org/search/sub";
readFromWeb(url);
}
}
You could additionally save it to file based on your needs or parse it using XML or HTML libraries.

Since Java 11 the most convenient way it to use java.net.http.HttpClient from the standard library.
Example:
HttpClient client = HttpClient.newBuilder()
.version(Version.HTTP_1_1)
.followRedirects(Redirect.NORMAL)
.connectTimeout(Duration.ofSeconds(20))
.proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))
.authenticator(Authenticator.getDefault())
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("httpss://foo.com/"))
.timeout(Duration.ofMinutes(2))
.GET()
.build();
HttpResponse<String> response = client.send(request, BodyHandlers.ofString());
System.out.println(response.statusCode());
System.out.println(response.body());

I use the following code for my API:
try {
URL url = new URL("https://stackoverflow.com/questions/6159118/using-java-to-pull-data-from-a-webpage");
InputStream content = url.openStream();
int c;
while ((c = content.read())!=-1) System.out.print((char) c);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException ie) {
ie.printStackTrace();
}
You can catch the characters and convert them to string.