Develop Reference

call javascript in wicket and set value from Wicket as a parameter

I have wicket component with onClick event where I'd like to run javascript code which:
reloads the page
after page has been reloaded, scroll down to the markupId which was clicked
I have to pass as parameter the "markupId" value from wicket to javascript to find out to which position should I scroll down
WicketComponent.java
MyPanel div = new MyPanel("div");
div.add(new AjaxEventBehavior("click") {
#Override
protected void onEvent(AjaxRequestTarget target) {
// some requests...
String markupId = div.getMarkupId();
target.appendJavaScript("window.location.reload();");
target.appendJavaScript(jsReload(markupId));
}
div.add(AttributeModifier.replace("onclick", "clicked('" + div.getMarkupId() + "');"));
#Override
public void renderHead(IHeaderResponse response) {
super.renderHead(response);
response.render(JavaScriptReferenceHeaderItem.forReference(new JavaScriptResourceReference(this.getClass(), "script.js")));
}
WicketComponent.html
<div wicket:id="div" onclick="clicked('markupId');">Text</div>
script.js
function clicked(markupId) {
window.location.reload();
}
document.addEventListener("DOMContentLoaded", function (event) {
let elementOffset = $("#{markupId}").offset().top; // how to pass here markupId parameter from wicket ?
let windowOffset = $(window).scrollTop();
window.scrollTo(0, elementOffset- windowOffset);
});
how to pass parameter "markupId" in javascript file which was attached in renderHead() or may be there is another solution for this ? I'll appreciate any help. Thanks!

you should solve your problem using location hash as described here:
Can we have code after location.reload(true)?
for the hash value use a fixed markup id for your component, something like:
div.setMarkupId("myMarkupId");
div.add(new AjaxEventBehavior("click") {
#Override
protected void onEvent(AjaxRequestTarget target) {
// some requests...
String markupId = div.getMarkupId();
target.appendJavaScript("window.location.hash = 'myMarkupId'");
target.appendJavaScript("window.location.reload();");
//that's it! no other js is needed
}
}
I haven't tried it but after page reloading it should scroll down to your component.

Can't parse the whole html page for some reason using jsoup

I'm trying to get the whole body page from youtube.com but only get a quarter of it for weird reasons
can somebody help me out here?
heres the code:
private static String data;
#Override
protected Void doInBackground(Void... voids) {
try {
Document doc = Jsoup.connect("https://www.youtube.com/results?search_query=Mettalica").get();
data = doc.body().html();
}
catch (IOException e) {
e.printStackTrace();
}
}
#Override
protected void onPostExecute(Void aVoid) {
//basically sysout the html results of the youtube search
super.onPostExecute(aVoid);
Log.d(TAG, data);
}

I think Doc Object has full HTML as well, you need to dig deeper to look for a better way but doc.outerHtml() should do the Job for you. Below SS also illustrates this object's state in Debug mode To compare with View Source of URL

Concurrency for recursive webcrawler-algorithm in Java

I wrote a program in Java to find all pages of a website, starting with the URL of the startpage (using Jsoup as webcrawler). It is ok for small websites but too slow for sites with 200 or more pages:
public class SiteInspector {
private ObservableSet<String> allUrlsOfDomain; // all URLS found for site
private Set<String> toVisit; // pages that were found but not visited yet
private Set<String> visited; // URLS that were visited
private List<String> invalid; // broken URLs
public SiteInspector() {...}
public void getAllWebPagesOfSite(String entry) //entry must be startpage of a site
{
toVisit.add(entry);
allUrlsOfDomain.add(entry);
while(!toVisit.isEmpty())
{
String next = popElement(toVisit);
getAllLinksOfPage(next); //expensive
toVisit.remove(next);
}
}
public void getAllLinksOfPage(String pageURL) {
try {
if (urlIsValid(pageURL)) {
visited.add(pageURL);
Document document = Jsoup.connect(pageURL).get(); //connect to pageURL (expensive network operation)
Elements links = document.select("a"); //get all links from page
for(Element link : links)
{
String nextUrl = link.attr("abs:href"); // "http://..."
if(nextUrl.contains(new URL(pageURL).getHost())) //ignore URLs to external hosts
{
if(!isForbiddenForCrawlers(nextUrl)) // URLS forbidden by robots.txt
{
if(!visited.contains(nextUrl))
{
toVisit.add(nextUrl);
}
}
allUrlsOfDomain.add(nextUrl);
}
}
}
else
{
invalid.add(pageURL); //URL-validation fails
}
}
catch (IOException e) {
e.printStackTrace();
}
}
private boolean isForbiddenForCrawlers(String url){...}
private boolean urlIsValid(String url) {...}
public String popElement(Set<String> set) {...}
I know I have to run the expensive network-operation in extra threads.
Document document = Jsoup.connect(pageURL).get(); //connect to pageURL
My problem is that I have no idea how to properly outsource this operation while keeping the sets consistent (how to synchronize?). If possible I want to use a ThreadPoolExecutor to control the amount of threads that is getting started during the process. Do you guys have an idea how to solve this? Thanks in advance.

To use threads and also keep the sets consistent, you just need to create a thread that receives the variable you want to add to the Set but created empty, so the thread fills it when done and then adds it to the Set.
A simple example of that could be:
Main.class
for (String link : links) {
String validUrl = null;
taskThread = new Thread( new WebDownloadThreadHanlder(link, validUrl, barrier));
taskThread.start();
if (validUrl != null) {
allUrlsOfDomain.add(validUrl);
}
}
barrier.acquireUninterruptibly(links.size());
WebDownloadThreadHandler.class
public class WebDownloadThreadHandler implements Runnable {
private String link;
private String validUrl;
private Semaphore barrier;
public ScopusThreadHandler(String link, String validUrl, Semaphore barrier) {
this.link = link;
this.validUrl = null;
this.barrier = barrier;
}
public void run () {
try {
Document document = Jsoup.connect(this.link).userAgent("Mozilla/5.0");
Elements elements = document.select(YOUR CSS QUERY);
/*
YOUR JSOUP CODE GOES HERE, AND STORE THE VALID URL IN: this.validUrl = THE VALUE YOU GET;
*/
} catch (IOException) {
e.printStackTrace();
}
this.barrier.release();
}
}
What you are doing here is creating a thread for every web you want to get all the links from, and storing them into variables, if you want to retrieve more than one lvalid link from every page, you can do it using a Set and adding it a to a global set (appending it). The thing is that to keep your code consistent you need to store the retrieved values in the variable you pass the thread as argument using THIS keyword.
Hope it helps! If you need anything else feel free to ask me!

Using jquery post to start a new java thread via a Http Servlet

I have a webpage with a start/pause button (controlling and xml extractor) which, when clicked, executes the following jquery function:
function start() {
// Posts to the start servlet
$.post("servlets/start", function(data) {
update();
});
}
I have a web.xml file mapping the url 'servlets/start' to a HttpServlet named 'StartServlet.java'. The doPost method of this servlet is supposed to start an xml extractor on a new thread or, if an extractor is already running it should pause it. That is all. The doPost method just calls startExtractor() as shown below.
private void newExtractor() {
ArrayList<URL> urls = null;
try {
String path = this.getServletContext().getRealPath(
"internalLinks.txt");
urls = GlobalUtils.getLinks(path);
extractor = new Extractor(urls);
thread = new Thread(extractor);
fillContextPool(extractor, false);
} catch (IOException e) {
System.out.println("Failed to start");
}
}
private void startExtractor() {
if (thread == null) {
newExtractor();
thread.run();
} else {
extractor.togglePause();
fillContextPool(extractor, stopForced);
}
}
The problem is, that the jquery post does not complete until the extractor does, meaning the button cannot be clicked again until the extractor has finished; essentially making it impossible to actually pause the extractor.
Any ideas as to how to make the post complete as son as the extractor is started and not have to wait until it has finished?
Thanks in advance!

A thread is started using the start() method. Not the run() method.

Implementing AJAX and database operations with Wicket

I'm trying to add AJAX to my project.
I have a link and a boolean variable named hasEngagement in my Wicket page. I want my link to produce a JavaScript informational warning if the boolean value is true, or perform a database operation otherwise. Here's my code:
Link myLink = new Link("mylink"){
#Override
onSubmit(){
if(hasEngagement)
//ajax operation
else
// database operation
}
};

You need to use an AjaxLink: http://wicket.apache.org/apidocs/1.4/org/apache/wicket/ajax/markup/html/AjaxLink.html
And override onClick
cheers
Lee

Also you can assign your message to a Feedback message. And of course use AjaxLink

AjaxLink myLink = new AjaxLink("mylink") {
#Override
public void onClick(AjaxRequestTarget target) {
if (hasEngagement) {
target.appendJavascript("alert('information warning');");
} else {
// database operation
}
};