I am trying to create a BitBucket plugin to get the repository structure and print it out in a structured format. The plugin creates a button on the repo page and when clicked it connects with a servlet to produce an output, however I cannot get my formatting code to work.
E.g
Instead of:
Folder 1
File 1
File 2
I want it to indent children:
Folder 1
File 1
File 2
I currently have a JS file which controls the button and makes an ajax call to a Java file, and also passes the servlet URL including the parameters for the repo (Project, Repo).
In my Java file I have a doGet which gets the repo from the parameters and uses a custom contentTreeCallback() to get the files within the repo in order to print them out, using callback.getFiles(). Within this same Java file, I have defined a node class which creates a linked hash map which takes each file, splits it into components, and with a recursive loop appends children to nested lists in order to create the file structure. This should work, however my custom contentTreeCallback() gets a string rather than the file array it needs to return. I cannot figure out what changes I need to make to get this to work. I'm guessing I either adjust the callback to get the files or I move the node class functionality into the callback class. I would prefer the second option since this class already splits the string, it seems a bit redundant to do it twice.
The servlet java class:
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
// Get values from the URL
projectName= req.getParameter("project");
repoName = req.getParameter("repository");
repo = repositoryService.getBySlug(projectName, repoName);
// ByteArrayOutputStream out = new ByteArrayOutputStream();
MyContentTreeCallback callback = new MyContentTreeCallback();
PageRequestImpl pr = new PageRequestImpl(0, 1000);
// Get information from the defined location, store in ByteArrayOutputStream
contentService.streamDirectory(repo, "Master", "", true, callback, pr);
resp.setContentType("text/html");
resp.getWriter().print("<html><body><p>Repository: " + repo.getName() + "</p>");
Node root = new Node(null);
for(int i = 0; i < callback.getFiles().size(); i++) {
root.add(callback.getFiles().get(i));
}
root.writeTo(resp.getWriter());
resp.getWriter().print("</body></html>");
}
static final class Node {
final String name;
final Map<String, Node> children = new LinkedHashMap<>();
Node(String name) {
this.name = name;
}
void add(File file) {
Node n = this;
for(String component: file.getPath().getComponents())
n = n.children.computeIfAbsent(component, Node::new);
}
void writeTo(Appendable w) throws IOException {
if(name != null) w.append("<li><a href='/'>").append(name).append("</a></li>\n");
if(!children.isEmpty()) {
w.append("<ul>\n");
for(Node ch: children.values()) ch.writeTo(w);
w.append("</ul>\n");
}
}
}
And the custom callback class:
public class MyContentTreeCallback extends AbstractContentTreeCallback {
ArrayList<File> files = new ArrayList<File>();
ContentTreeSummary fileSummary;
public MyContentTreeCallback() {
}
#Override
public void onEnd(#Nonnull ContentTreeSummary summary) {
fileSummary = summary;
}
#Override
public void onStart(#Nonnull ContentTreeContext context) {
System.out.print("On start");
}
#Override
public boolean onTreeNode(#Nonnull ContentTreeNode node) {
String filePath = "";
if (node.getPath().getComponents().length>1) {
for(int i=0;i<node.getPath().getComponents().length;i++) {
filePath+=node.getPath().getComponents()[i]+"/";
//filePath=filePath.substring(0,filePath.length() - 1)
}
}
else {
filePath+=node.getPath().getName();
}
String lastChar = String.valueOf(filePath.charAt(filePath.length() - 1));
if(lastChar.equals("/")){ filePath=filePath.substring(0,filePath.length() -
1); }
files.add(filePath);
return true;
}
public ArrayList<File> getFiles(){
return files;
}
}
files.add(filePath); Is where the issue is in the callback class.
I'm sure it's simpler than I am making it out to be... Thanks for any help you can give
Related
I have been struggling all day with this seemlesly unimportant issue, but I need to get it sorted. I will post all my code and explain it. I need to parse a CSV into a HashMap with structure {key, value} where value is an ArrayList of ArrayLists of Strings.
HashMap<String, ArrayList<ArrayList<String>>>
In the back, using AsyncTask, I get the data correctly into the map using a CSV document where I have the information stored. Here is my AsyncTask, just in case, but I can 99% say it works just fine.
public class ReadAllPaths extends AsyncTask<Integer, Double, HashMap<String, ArrayList<ArrayList<String>>>> {
public interface AsyncResponse {
void processFinish(HashMap<String, ArrayList<ArrayList<String>>> output);
}
public static final String TAG = "ReadAllPaths";
public AsyncResponse delegate;
#SuppressLint("StaticFieldLeak")
private final Context context;
public ReadAllPaths(Context context, AsyncResponse delegate) {
this.context = context;
this.delegate = delegate;
}
#Override
protected HashMap<String, ArrayList<ArrayList<String>>> doInBackground(Integer... allPaths) {
return readData(allPaths[0]);
}
#Override
protected void onProgressUpdate(Double... progress) {
DecimalFormat df = new DecimalFormat();
df.setMaximumFractionDigits(2);
}
#Override
protected void onPostExecute(HashMap<String, ArrayList<ArrayList<String>>> result) {
delegate.processFinish(result);
}
private HashMap<String, ArrayList<ArrayList<String>>> readData(int allPaths) {
InputStream is = context.getResources().openRawResource(allPaths);
BufferedReader reader = new BufferedReader(
new InputStreamReader(is, StandardCharsets.UTF_8));
String line = "";
HashMap<String, ArrayList<ArrayList<String>>> all_paths = new HashMap<>();
ArrayList<ArrayList<String>> value_date;
try {
while ((line = reader.readLine()) != null) {
if (isCancelled()) break;
value_date = new ArrayList<>();
// Split the line into different tokens
String[] tokens = line.split(";");
String key_date = tokens[0];
value_date.add(new ArrayList<>(Arrays.asList(tokens[1].split("\\s*,\\s*"))));
value_date.add(new ArrayList<>(Arrays.asList(tokens[2].split("\\s*,\\s*"))));
value_date.add(new ArrayList<>(Arrays.asList(tokens[3].split("\\s*,\\s*"))));
all_paths.put(key_date, value_date);
}
} catch (IOException e) {
Log.d(TAG, "Error " + line, e);
e.printStackTrace();
return null;
}
return all_paths;
}
}
In my main Activity I make a call to it and its onPostExecute method. Again, posting it just in case,
ReadAllPaths readAllPaths = (ReadAllPaths) new ReadAllPaths(this, output -> {
// Here you will receive the result fired from async class
// of onPostExecute(result) method.
Log.d(TAG, "processFinish: Process finished!");
everyPath = output;
Log.d(TAG, "onCreate: output: " + output);
}).execute(R.raw.all_paths);
What is strange is wat happens now. I can make a log.d, or print or whatever and print my output, and it will print my data perfectly. On the next stage, I need to "dynamically" access the map, so I use this simple function (level is just a avlue from 0 to 2):
private ArrayList<String> getPath(String date, int level) {
// Example of structure
// {"19-05-2022": [[path1], [path2], [path3]}]}
return everyPath.get(date).get(level);
}
Now this works perfeclty everytime except for the first element in my CSV. I know maps don't have an order, so I tried changing my CSV and the exception keeps occurring on the first line of my CSV though it is correctly displayed when I do my log.d. Also I have tried checking if key exists via:
everyPath.containsKey(date);
And also returns false only on the first row of my CSV. At first I thought this could be due to dealing with headers in the CSV, but again, the data is correctly displayed when I print the output of my AsyncTask. Only the part where I need to fetch the data is not working.
Also I found a way around, which is duplicating my first row of the CSV, but I would rather understand what is going on here.
Thank you!
I wrote a program in Java to find all pages of a website, starting with the URL of the startpage (using Jsoup as webcrawler). It is ok for small websites but too slow for sites with 200 or more pages:
public class SiteInspector {
private ObservableSet<String> allUrlsOfDomain; // all URLS found for site
private Set<String> toVisit; // pages that were found but not visited yet
private Set<String> visited; // URLS that were visited
private List<String> invalid; // broken URLs
public SiteInspector() {...}
public void getAllWebPagesOfSite(String entry) //entry must be startpage of a site
{
toVisit.add(entry);
allUrlsOfDomain.add(entry);
while(!toVisit.isEmpty())
{
String next = popElement(toVisit);
getAllLinksOfPage(next); //expensive
toVisit.remove(next);
}
}
public void getAllLinksOfPage(String pageURL) {
try {
if (urlIsValid(pageURL)) {
visited.add(pageURL);
Document document = Jsoup.connect(pageURL).get(); //connect to pageURL (expensive network operation)
Elements links = document.select("a"); //get all links from page
for(Element link : links)
{
String nextUrl = link.attr("abs:href"); // "http://..."
if(nextUrl.contains(new URL(pageURL).getHost())) //ignore URLs to external hosts
{
if(!isForbiddenForCrawlers(nextUrl)) // URLS forbidden by robots.txt
{
if(!visited.contains(nextUrl))
{
toVisit.add(nextUrl);
}
}
allUrlsOfDomain.add(nextUrl);
}
}
}
else
{
invalid.add(pageURL); //URL-validation fails
}
}
catch (IOException e) {
e.printStackTrace();
}
}
private boolean isForbiddenForCrawlers(String url){...}
private boolean urlIsValid(String url) {...}
public String popElement(Set<String> set) {...}
I know I have to run the expensive network-operation in extra threads.
Document document = Jsoup.connect(pageURL).get(); //connect to pageURL
My problem is that I have no idea how to properly outsource this operation while keeping the sets consistent (how to synchronize?). If possible I want to use a ThreadPoolExecutor to control the amount of threads that is getting started during the process. Do you guys have an idea how to solve this? Thanks in advance.
To use threads and also keep the sets consistent, you just need to create a thread that receives the variable you want to add to the Set but created empty, so the thread fills it when done and then adds it to the Set.
A simple example of that could be:
Main.class
for (String link : links) {
String validUrl = null;
taskThread = new Thread( new WebDownloadThreadHanlder(link, validUrl, barrier));
taskThread.start();
if (validUrl != null) {
allUrlsOfDomain.add(validUrl);
}
}
barrier.acquireUninterruptibly(links.size());
WebDownloadThreadHandler.class
public class WebDownloadThreadHandler implements Runnable {
private String link;
private String validUrl;
private Semaphore barrier;
public ScopusThreadHandler(String link, String validUrl, Semaphore barrier) {
this.link = link;
this.validUrl = null;
this.barrier = barrier;
}
public void run () {
try {
Document document = Jsoup.connect(this.link).userAgent("Mozilla/5.0");
Elements elements = document.select(YOUR CSS QUERY);
/*
YOUR JSOUP CODE GOES HERE, AND STORE THE VALID URL IN: this.validUrl = THE VALUE YOU GET;
*/
} catch (IOException) {
e.printStackTrace();
}
this.barrier.release();
}
}
What you are doing here is creating a thread for every web you want to get all the links from, and storing them into variables, if you want to retrieve more than one lvalid link from every page, you can do it using a Set and adding it a to a global set (appending it). The thing is that to keep your code consistent you need to store the retrieved values in the variable you pass the thread as argument using THIS keyword.
Hope it helps! If you need anything else feel free to ask me!
I'm using the upload component of vaadin(7.1.9), now my trouble is that I'm not able to restrict what kind of files that can be sent with the upload component to the server, but I haven't found any API for that purpose. The only way is that of discarding file of wrong types after the upload.
public OutputStream receiveUpload(String filename, String mimeType) {
if(!checkIfAValidType(filename)){
upload.interruptUpload();
}
return out;
}
Is this a correct way?
No, its not the correct way. The fact is, Vaadin does provide many useful interfaces that you can use to monitor when the upload started, interrupted, finished or failed. Here is a list:
com.vaadin.ui.Upload.FailedListener;
com.vaadin.ui.Upload.FinishedListener;
com.vaadin.ui.Upload.ProgressListener;
com.vaadin.ui.Upload.Receiver;
com.vaadin.ui.Upload.StartedListener;
Here is a code snippet to give you an example:
#Override
public void uploadStarted(StartedEvent event) {
// TODO Auto-generated method stub
System.out.println("***Upload: uploadStarted()");
String contentType = event.getMIMEType();
boolean allowed = false;
for(int i=0;i<allowedMimeTypes.size();i++){
if(contentType.equalsIgnoreCase(allowedMimeTypes.get(i))){
allowed = true;
break;
}
}
if(allowed){
fileNameLabel.setValue(event.getFilename());
progressBar.setValue(0f);
progressBar.setVisible(true);
cancelButton.setVisible(true);
upload.setEnabled(false);
}else{
Notification.show("Error", "\nAllowed MIME: "+allowedMimeTypes, Type.ERROR_MESSAGE);
upload.interruptUpload();
}
}
Here, allowedMimeTypes is an array of mime-type strings.
ArrayList<String> allowedMimeTypes = new ArrayList<String>();
allowedMimeTypes.add("image/jpeg");
allowedMimeTypes.add("image/png");
I hope it helps you.
Can be done.
You can add this and it will work (all done by HTML 5 and most browsers now support accept attribute) - this is example for .csv files:
upload.setButtonCaption("Import");
JavaScript.getCurrent().execute("document.getElementsByClassName('gwt-FileUpload')[0].setAttribute('accept', '.csv')");
I think it's better to throw custom exception from Receiver's receiveUpload:
Upload upload = new Upload(null, new Upload.Receiver() {
#Override
public OutputStream receiveUpload(String filename, String mimeType) {
boolean typeSupported = /* do your check*/;
if (!typeSupported) {
throw new UnsupportedImageTypeException();
}
// continue returning correct stream
}
});
The exception is just a simple custom exception:
public class UnsupportedImageTypeException extends RuntimeException {
}
Then you just simply add a listener if the upload fails and check whether the reason is your exception:
upload.addFailedListener(new Upload.FailedListener() {
#Override
public void uploadFailed(Upload.FailedEvent event) {
if (event.getReason() instanceof UnsupportedImageTypeException) {
// do your stuff but probably don't log it as an error since it's not 'real' error
// better would be to show sth like a notification to inform your user
} else {
LOGGER.error("Upload failed, source={}, component={}", event.getSource(), event.getComponent());
}
}
});
public static boolean checkFileType(String mimeTypeToCheck) {
ArrayList allowedMimeTypes = new ArrayList();
allowedMimeTypes.add("image/jpeg");
allowedMimeTypes.add("application/pdf");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.wordprocessingml.document");
allowedMimeTypes.add("image/png");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.presentationml.presentation");
allowedMimeTypes.add("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
for (int i = 0; i < allowedMimeTypes.size(); i++) {
String temp = allowedMimeTypes.get(i);
if (temp.equalsIgnoreCase(mimeTypeToCheck)) {
return true;
}
}
return false;
}
I am working with Vaadin 8 and I there is no change in Upload class.
FileUploader receiver = new FileUploader();
Upload upload = new Upload();
upload.setAcceptMimeTypes("application/json");
upload.setButtonCaption("Open");
upload.setReceiver(receiver);
upload.addSucceededListener(receiver);
FileUploader is the class that I created that handles the upload process. Let me know if you need to see the implementation.
I want to download a CSV file using Wicket, by implementing an AbstractResource. It looks something like this:
public class ExportCsvFileResource extends AbstractResource
{
#Override
protected AbstractResource.ResourceResponse newResourceResponse(IResource.Attributes attributes)
{
AbstractResource.ResourceResponse resourceResponse = new AbstractResource.ResourceResponse();
resourceResponse.setContentType("text/csv");
resourceResponse.setFileName("exported-contacts-file.csv");
resourceResponse.setTextEncoding("utf-8");
resourceResponse.setWriteCallback(new AbstractResource.WriteCallback()
{
#Override
public void writeData(IResource.Attributes attributes) throws IOException
{
OutputStream stream = attributes.getResponse().getOutputStream();
generateContentInBatches(stream);
}
});
return resourceResponse;
}
private void generateContentInBatches(OutputStream stream)
{
int numberOfChunks=//...
for (int i=0; i<numberOfChunks; i++)
{
byte[] contentChunk = retrieveContentFromBackend(i);
IOUtils.write(contentChunk, stream);
}
}
}
The problem is that, while the content is being generated with the retrieveContentFromBackend function (which is quite time consuming), the user interface is unresponsive. I click the buttons etc. but nothing happens, only after the file is done being generate can I use the interface again.
How do I avoid blocking the user interface while the file is being generated gradually?
Take a look at RequestMapperApplication and MapperDemoResourceReference from wicket-examples.
You can mount resource references:
mountResource("/print/${sheet}/${format}", new MapperDemoResourceReference());
To load such a resource without blocking the page, you'll have to render a link which triggers the resource directly:
add(new WebMarkupContainer("link")
{
#Override
protected void onComponentTag(ComponentTag tag)
{
super.onComponentTag(tag);
PageParameters parameters = new PageParameters();
parameters.add("sheet", "sheet1");
parameters.add("format", "A4");
tag.put("href", urlFor(new MapperDemoResourceReference(), parameters));
}
});
Here is an example of lazy loading:
http://www.wicket-library.com/wicket-examples/ajax/lazy-loading?1
I don't know how this works with your AbstractResource object but this should get you in the right direction.
I have written a client and server application who are connected with sockets.
The server is multithreaded and can hold more than one client.
Now am i using in my client application a jTree model, and on my server i have a folder with fictitious project folders and files in it.
First this application only was client sided with a database on a server, now i want it to fully operate over a socket connection.
The only problem is that the jTree model is created on the server. So when the program starts the client asks the server to create a jTree model of the project folder the user will use in the client application with its files in it.
The Server will send a respone with a jTree model in it and the client puts this model in the jTree swing component.
The jTree is succesfully shown to the user (so the Tree model is succesfully sended over the socket connection), but when the user wants to expand the first projectfolder map: the jTree shows for example: 'testProject' as folder.
And for example the folder testProject contains some files:
testProject.exe
bin
test.java
test2.java
src
As you see in the above example the jTree should show the above tree to the user when the user double clicks on 'testProject' in the jTree.
Now my problem
I think the problem is that the server only creates the model of the first folder.
Like it only sends a model with 'projectFolder' in it.
But i want the above structure send to the user so the user can use the jTree to download the files from the server. (something i will implement later).
How can i get a full model from the server, that the jTree can show a full model to the user with the projectfolder and all his files.
To make everything a little more clear:
My code
This fragment of code i use on my client to invoke the method that created the jTree model on the server.
public ProjectTreeModel getTreeModel(String projectName) throws ClassNotFoundException {
try {
//Creating empty object to invoke te mehod on the server.
paramObject parameters = new paramObject();
parameters.string1 = projectName;
//Creating package to invoke the right method on the server.
Packet data = new Packet("treemodel.GetModel",parameters);
//sends in this case an empty object to the server and invokes the treemodel.GetModel method on the server.
oos.writeObject(data);
oos.flush();
ProjectTreeModel model = (ProjectTreeModel) ois.readObject();
return model;
} catch (IOException ex) {
ex.printStackTrace();
}
return null;
}
This piece of code i use on the server to create the projectModel and send it back as object to the client to return it as a jTree model in the previous code example.
if(data.getMethod().equals("treemodel.GetModel"))
{
//in this example i don't use this object, just ignore it.
paramObject parameters = (paramObject) data.getObject();
//Extract the information from User
String projectName = parameters.string1;
ProjectTreeModel treemodel = new ProjectTreeModel();
treemodel.newRootPath(projectName);
oos.writeObject(treemodel);
output.flush();
form.sendOutput("ProjectTreeModel succesfully sended to: " + Email);
}
And at last this is how i create the TreeModel.
import java.io.File;
import java.io.Serializable;
import javax.swing.event.TreeModelListener;
import javax.swing.tree.TreeModel;
import javax.swing.tree.TreePath;
public class ProjectTreeModel implements TreeModel, Serializable
{
File f;
private static String rootPath = "src/authorsystem/Projects";
public static void newRootPath(String newRoot)
{
String desktopPath = System.getProperty("user.home") + "\\Desktop";
desktopPath.replace("\\", "/");
rootPath = desktopPath + "\\projectfolders";
}
#Override
public Object getRoot() {
//userName = LogInForm.ingelogt;
f = new File(rootPath);
return f;
}
#Override
public Object getChild(Object parent, int index) {
File pathName = (File) parent;
String[] fileNames = pathName.list();
return new File(pathName.getPath(), fileNames[index]);
}
#Override
public int getChildCount(Object parent) {
File pathName = (File) parent;
if(pathName.list() != null)
{
String[] fileNames = pathName.list();
return fileNames.length;
}
else
{
return 0;
}
}
#Override
public boolean isLeaf(Object node) {
return ((File)node).isFile();
}
#Override
public void valueForPathChanged(TreePath path, Object newValue) {
}
#Override
public int getIndexOfChild(Object parent, Object child) {
File pathName = (File) parent;
File childName = (File) child;
if(pathName.isDirectory())
{
String[] fileNames = pathName.list();
for (int i = 0; i < fileNames.length; i++) {
if(pathName.compareTo(childName) == 0)
{
return i;
}
}
}
return -1;
}
#Override
public void addTreeModelListener(TreeModelListener l) {
}
#Override
public void removeTreeModelListener(TreeModelListener l) {
}
}
I hope somebody can help me out, or just help me with ideas how to solve this problem.
Ok just to close this topic in case if someone find this.
Its really long ago i worked on this but this is how we managed it:
For the treeModel we made our custom nodes. Instead of sending a JTreeModel object trough a socket connection we only send the data that must be shown in the tree.
After the client received the data from the server we iterated trough all the data and checked if the node is a parent or child.
This can be done basically with the overrided TreeModel class code snipper above in my Question.