I have an Iterator that should be being populated by the FileUtils.iterateFiles() function using an instance of IOFileFilter to sort through and return an iterator of files matching the title "sound".
The problem is, even though i can see the accept(File file) override in the IOFileFilter returning true (via breakpoint), when i query iterator.hasNext(), I get false, and debug shows that the iterator is empty. Can anybody spot any possible causes? Or do Iterators and IOFileFiltters not behave in the way in which i think they do?
Code below.
private IOFileFilter findSoundsDir = new IOFileFilter()
{
#Override
public boolean accept(File file) {
return file.getName().equals("sounds"); //Breakpoint here gets hit
}
#Override
public boolean accept(File file, String s) {
return s.equals("sounds");
}
};
public void generateSoundEmbeds(String baseDir)
{
File searchingDir = new File(baseDir).getParentFile().getParentFile();
System.out.println("searchingDir: " + searchingDir.getAbsolutePath());
Iterator<File> soundsIterator = (Iterator<File>)FileUtils.iterateFiles(searchingDir, findSoundsDir ,TrueFileFilter.INSTANCE);
if(soundsIterator.hasNext()) //however this check returns false, despite IOFileFilter returning true at least once previously
{
//we never get here
}else{
System.out.println("found no file");
}
}
Related
I am trying to create a BitBucket plugin to get the repository structure and print it out in a structured format. The plugin creates a button on the repo page and when clicked it connects with a servlet to produce an output, however I cannot get my formatting code to work.
E.g
Instead of:
Folder 1
File 1
File 2
I want it to indent children:
Folder 1
File 1
File 2
I currently have a JS file which controls the button and makes an ajax call to a Java file, and also passes the servlet URL including the parameters for the repo (Project, Repo).
In my Java file I have a doGet which gets the repo from the parameters and uses a custom contentTreeCallback() to get the files within the repo in order to print them out, using callback.getFiles(). Within this same Java file, I have defined a node class which creates a linked hash map which takes each file, splits it into components, and with a recursive loop appends children to nested lists in order to create the file structure. This should work, however my custom contentTreeCallback() gets a string rather than the file array it needs to return. I cannot figure out what changes I need to make to get this to work. I'm guessing I either adjust the callback to get the files or I move the node class functionality into the callback class. I would prefer the second option since this class already splits the string, it seems a bit redundant to do it twice.
The servlet java class:
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
// Get values from the URL
projectName= req.getParameter("project");
repoName = req.getParameter("repository");
repo = repositoryService.getBySlug(projectName, repoName);
// ByteArrayOutputStream out = new ByteArrayOutputStream();
MyContentTreeCallback callback = new MyContentTreeCallback();
PageRequestImpl pr = new PageRequestImpl(0, 1000);
// Get information from the defined location, store in ByteArrayOutputStream
contentService.streamDirectory(repo, "Master", "", true, callback, pr);
resp.setContentType("text/html");
resp.getWriter().print("<html><body><p>Repository: " + repo.getName() + "</p>");
Node root = new Node(null);
for(int i = 0; i < callback.getFiles().size(); i++) {
root.add(callback.getFiles().get(i));
}
root.writeTo(resp.getWriter());
resp.getWriter().print("</body></html>");
}
static final class Node {
final String name;
final Map<String, Node> children = new LinkedHashMap<>();
Node(String name) {
this.name = name;
}
void add(File file) {
Node n = this;
for(String component: file.getPath().getComponents())
n = n.children.computeIfAbsent(component, Node::new);
}
void writeTo(Appendable w) throws IOException {
if(name != null) w.append("<li><a href='/'>").append(name).append("</a></li>\n");
if(!children.isEmpty()) {
w.append("<ul>\n");
for(Node ch: children.values()) ch.writeTo(w);
w.append("</ul>\n");
}
}
}
And the custom callback class:
public class MyContentTreeCallback extends AbstractContentTreeCallback {
ArrayList<File> files = new ArrayList<File>();
ContentTreeSummary fileSummary;
public MyContentTreeCallback() {
}
#Override
public void onEnd(#Nonnull ContentTreeSummary summary) {
fileSummary = summary;
}
#Override
public void onStart(#Nonnull ContentTreeContext context) {
System.out.print("On start");
}
#Override
public boolean onTreeNode(#Nonnull ContentTreeNode node) {
String filePath = "";
if (node.getPath().getComponents().length>1) {
for(int i=0;i<node.getPath().getComponents().length;i++) {
filePath+=node.getPath().getComponents()[i]+"/";
//filePath=filePath.substring(0,filePath.length() - 1)
}
}
else {
filePath+=node.getPath().getName();
}
String lastChar = String.valueOf(filePath.charAt(filePath.length() - 1));
if(lastChar.equals("/")){ filePath=filePath.substring(0,filePath.length() -
1); }
files.add(filePath);
return true;
}
public ArrayList<File> getFiles(){
return files;
}
}
files.add(filePath); Is where the issue is in the callback class.
I'm sure it's simpler than I am making it out to be... Thanks for any help you can give
I would like to just crawl with crawler4j, certain URLs which have a certain prefix.
So for example, if an URL starts with http://url1.com/timer/image it is valid. E.g.: http://url1.com/timer/image/text.php.
This URL is not valid: http://test1.com/timer/image
I tried to implement it like that:
public boolean shouldVisit(Page page, WebURL url) {
String href = url.getURL().toLowerCase();
String adrs1 = "http://url1.com/timer/image";
String adrs2 = "http://url2.com/house/image";
if (!(href.startsWith(adrs1)) || !(href.startsWith(adrs2))) {
return false;
}
if (filters.matcher(href).matches()) {
return false;
}
for (String crawlDomain : myCrawlDomains) {
if (href.startsWith(crawlDomain)) {
return true;
}
}
return false;
}
However, it does not seem that this works, because the crawler also visits other URLs.
Any recommendation what I could so?
I appreciate your answer!
Basically you can have an array of prefixes which holds allowed URLs which you want to crawl. And inside your method just travers the array return true if only it machetes with any of your allowed prefix. That means you dont have to list any domains which you don't want to crawl.
public boolean shouldVisit(Page page, WebURL url) {
String href = url.getURL().toLowerCase();
// prefixes that you want to crawl
String allowedPrefixes[] = {"http://url1.com", "http://url2.com"};
for (String allowedPrefix : allowedPrefixes) {
if (href.startsWith(allowedPrefix)) {
return true;
}
}
return false;
}
Your code is not working because your condition is incorrect:
(!(href.startsWith(adrs1)) || !(href.startsWith(adrs2))
Another reason is you might not have configured crawlerDomains. It is configured during startup of your application by calling CrawlController#setCustomData(crawler1Domains);
Look at sample source code of crawler4j, crawlerDomains are set here: MultipleCrawlerController.java#79
Look at the below code. it may help you.
public boolean shouldVisit(Page page,WebURL url) {
String href = url.getURL().toLowerCase();
String adrs1 = "http://url1.com/timer/image";
String adrs2 = "http://url2.com/house/image";
return !FILTERS.matcher(href).matches() && (href.startsWith(adrs1) || href.startsWith(adrs2));
}
I'm using the upload component of vaadin(7.1.9), now my trouble is that I'm not able to restrict what kind of files that can be sent with the upload component to the server, but I haven't found any API for that purpose. The only way is that of discarding file of wrong types after the upload.
public OutputStream receiveUpload(String filename, String mimeType) {
if(!checkIfAValidType(filename)){
upload.interruptUpload();
}
return out;
}
Is this a correct way?
No, its not the correct way. The fact is, Vaadin does provide many useful interfaces that you can use to monitor when the upload started, interrupted, finished or failed. Here is a list:
com.vaadin.ui.Upload.FailedListener;
com.vaadin.ui.Upload.FinishedListener;
com.vaadin.ui.Upload.ProgressListener;
com.vaadin.ui.Upload.Receiver;
com.vaadin.ui.Upload.StartedListener;
Here is a code snippet to give you an example:
#Override
public void uploadStarted(StartedEvent event) {
// TODO Auto-generated method stub
System.out.println("***Upload: uploadStarted()");
String contentType = event.getMIMEType();
boolean allowed = false;
for(int i=0;i<allowedMimeTypes.size();i++){
if(contentType.equalsIgnoreCase(allowedMimeTypes.get(i))){
allowed = true;
break;
}
}
if(allowed){
fileNameLabel.setValue(event.getFilename());
progressBar.setValue(0f);
progressBar.setVisible(true);
cancelButton.setVisible(true);
upload.setEnabled(false);
}else{
Notification.show("Error", "\nAllowed MIME: "+allowedMimeTypes, Type.ERROR_MESSAGE);
upload.interruptUpload();
}
}
Here, allowedMimeTypes is an array of mime-type strings.
ArrayList<String> allowedMimeTypes = new ArrayList<String>();
allowedMimeTypes.add("image/jpeg");
allowedMimeTypes.add("image/png");
I hope it helps you.
Can be done.
You can add this and it will work (all done by HTML 5 and most browsers now support accept attribute) - this is example for .csv files:
upload.setButtonCaption("Import");
JavaScript.getCurrent().execute("document.getElementsByClassName('gwt-FileUpload')[0].setAttribute('accept', '.csv')");
I think it's better to throw custom exception from Receiver's receiveUpload:
Upload upload = new Upload(null, new Upload.Receiver() {
#Override
public OutputStream receiveUpload(String filename, String mimeType) {
boolean typeSupported = /* do your check*/;
if (!typeSupported) {
throw new UnsupportedImageTypeException();
}
// continue returning correct stream
}
});
The exception is just a simple custom exception:
public class UnsupportedImageTypeException extends RuntimeException {
}
Then you just simply add a listener if the upload fails and check whether the reason is your exception:
upload.addFailedListener(new Upload.FailedListener() {
#Override
public void uploadFailed(Upload.FailedEvent event) {
if (event.getReason() instanceof UnsupportedImageTypeException) {
// do your stuff but probably don't log it as an error since it's not 'real' error
// better would be to show sth like a notification to inform your user
} else {
LOGGER.error("Upload failed, source={}, component={}", event.getSource(), event.getComponent());
}
}
});
public static boolean checkFileType(String mimeTypeToCheck) {
ArrayList allowedMimeTypes = new ArrayList();
allowedMimeTypes.add("image/jpeg");
allowedMimeTypes.add("application/pdf");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.wordprocessingml.document");
allowedMimeTypes.add("image/png");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.presentationml.presentation");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
for (int i = 0; i < allowedMimeTypes.size(); i++) {
String temp = allowedMimeTypes.get(i);
if (temp.equalsIgnoreCase(mimeTypeToCheck)) {
return true;
}
}
return false;
}
I am working with Vaadin 8 and I there is no change in Upload class.
FileUploader receiver = new FileUploader();
Upload upload = new Upload();
upload.setAcceptMimeTypes("application/json");
upload.setButtonCaption("Open");
upload.setReceiver(receiver);
upload.addSucceededListener(receiver);
FileUploader is the class that I created that handles the upload process. Let me know if you need to see the implementation.
i have used Apache FileUtils and IOFileFilter to list all files under a folder recursively excluding .svn folders. Here is the code i tried
File selectedFolder = new File(path);\\path to folder to list
final IOFileFilter dirs = new IOFileFilter() {
#Override
public boolean accept(File file, String s) {
return file.isDirectory();
}
#Override
public boolean accept(File file) {
// TODO Auto-generated method stub
if(file.getName().toLowerCase().equalsIgnoreCase(".svn")||file.getName().toLowerCase().contains(".svn"))
return false;
else return true;
}
};
filesList.addAll(FileUtils.listFiles(selectedFolder,dirs, TrueFileFilter.INSTANCE));
I am getting the error
java.lang.IllegalArgumentException: Parameter 'directory' is not a directory
at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545)
at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521)
Can anyone tell me where am going wrong. I feel there is something wrong with the filter used. I could not figure it out
Actually, FileFilterUtils contains a method called makeSVNAware that you could use. It returns a filter that ignores SVN directories. For example:
filesList.addAll(
FileUtils.listFiles(selectedFolder, TrueFileFilter.TRUE,
FileFilterUtils.makeSVNAware(null)));
Note that listFiles expects a file filter as its 2nd argument, and a dir filter as its 3rd. In your code they're the other way round. So if you wouldn't want to use makeSVNAware, your code would look something like this:
File selectedFolder = new File(path); // path to folder to list
final IOFileFilter dirs = new IOFileFilter() {
#Override
public boolean accept(File file, String s) {
return file.isDirectory();
}
#Override
public boolean accept(File file) {
return (!file.getName().toLowerCase().equalsIgnoreCase(".svn"));
}
};
// 2nd argument: TRUE filter, returning all files
// 3rd argument: dirs filter, returning all directories except those named .svn
filesList.addAll(FileUtils.listFiles(selectedFolder, TrueFileFilter.TRUE, dirs));
It looks like you have split the functionality into two functions.
The second one should also check for isDirectory() and the first one should also check the name.
I am trying to set the file filter for my JFileChooser. This is my code:
JFileChooser picker= new JFileChooser();
picker.setFileFilter(new FileNameExtensionFilter("txt"));
int pickerResult = picker.showOpenDialog(getParent());
if (pickerResult == JFileChooser.APPROVE_OPTION){
System.out.println("This works!");
}
if (pickerResult == JFileChooser.CANCEL_OPTION){
System.exit(1);
}
When I run my program, the file chooser comes up, but it won't let me pick any .txt files. Instead, it says this in the console:
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Extensions must be non-null and not empty
How do i fix this?
You need to add at least one extension as a second paramter. From the API:
FileNameExtensionFilter(String description, String... extensions)
Parameters:
description - textual description for the filter, may be null
extensions - the accepted file name extensions
Also if you want an specific files extensions and navigate thru folders you can try this:
JFileChooser fc = new JFileChooser(path);
fc.setFileSelectionMode(JFileChooser.FILES_ONLY);
fc.addChoosableFileFilter(new FileFilter () {
#Override
public String getDescription() {
return "DAT Files";
}
#Override
public boolean accept(File f) {
if (f.isDirectory())
return true;
return f.getName().endsWith(".dat");
}
});
fc.setAcceptAllFileFilterUsed(false);