Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
What is the best library for a Java web scraper? I know the following choices:
Selenium
HTMLUnit
Lobo browser
I need to select one option to build a scraper for one scalable project.
If you are scraping, why do you need a browser? Just doing basic cURL calls to a page and getting the response will give you what you need to do scraping.
This will help with scalability. If you want a browser then go for HTMLUnit as that would again help with scalability.
I was recently recommended Web Harvest, and thought it worked well out of the box, except for some issues around HTTP 500 response codes ...
Use jsoup, it works great to get the response from URL and then use the XPath Expression to parse data from the response. I've implemented this and it works great.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
So, I need to write software that automates some browser tasks for me. Nothing fancy (clicking and getting data from the webpage). I want to use Java. Can you point me in a direction where to look/start, like libraries?
Thanks in advance!
Cheers
It is written for the purpose of running automated tests against websites, but you could use it for this job too: Selenium.
However, before you try to automate an entire browser, check if the service you're connecting to has an API, because otherwise your script will definitely just break if the site updates its styles a bit.
Even if there is no API, check if the answers you are looking for are just sent in the HTML - if so, just load the page with JSoup and parse the answer right out, no need to start or script any browsers.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I built a Java game for a CS class. The code can be found here. The main class is Game.Java found here. I have a simple static website hosted on GoDaddy found here. I would like to allow a visitor to my website to play the game on a web browser I don't have a server yet, so I'll buy a basic
VPS through GoDaddy. Besides that, what's the best way to go about it? I think Applet is no longer supported by Chrome. Is Java Web Start the only way now? I would prefer if the visitor doesn't have to download anything. Any supporting material or YouTube video would be helpful as well.
Java is no longer supported by any modern browser, so the only way people will be able to play your game is to either include a client-side, JS/WASM-based library that sets up a complete JVM and then running the game inside of that on your page, or by giving people a link to your game's .jar file, so that they can download and run it on their own computer.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 7 years ago.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Improve this question
I got a mine craft server online (https://server.pro/control-panel/1446793/) and I need to click the restart button every 45-55 Minutes. How can I do this? Is it possible to Do using GC (google code)? If not do i use java? Is it even possible? I have no idea what to do. Any response is helpful (even if it says something I have done wrong :) )
If i do need to use JAVA what program would i use to make the script (If there is one)
I am not familiar with either of those languages, however I have made an autoclicker in C. It can be found here. If you are unwilling to use mine, there are many other autoclickers out there.
There are many ways to do it. You could write some javascript that is injected into the page by plugin. You could also do it in Java with a Robot although that would be more hacky than the first solution. Also the Selenium library may help you as it's used for writing tests that will click different things on the web page.
You don't need to click the button from the remote to restart the server.
You can do that with a plugin, for example auto-restart plugin
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I'm writing the code to access and download data from some web sites using HTMLUnit.
I was wondering whether a tool that could help me exists. I think maybe to a visual tool, something that lets you express the path you want to follow on the web sites visually and generates the Java code.
I'd like something for HTMLUnit, but I'm open to other solutions.
Writing screen-scraping code gets very repetitive :(
I used to work at screen-scraper. Not sure if it's quite what you're looking for, but it's definitely a powerful tool for building web-scraping scripts.
Not exactly visual tools, but maybe you could use something like JMeter or The Grinder for recording navigation scripts of web pages. They surely are easier to use than writing all the input requests by hand in HTMLUnit.
There is only one tool that does exactly this - and does it very very well - but it's highly expensive, since their technology is amazing. Some of the biggest companies uses this. I think you might be able to get a free trail, if you want to try it out. http://kapowsoftware.com/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I have downloaded facebook-java-api-3.0.2-bin from
http://code.google.com/p/facebook-java-api/
for connecting to face book.I want to use this api in my java application to connect Facebook.I am developing a desktop application for this purpose.I have created account on Face book and registered as a developer.
Is it possible to connect Facebook from my java application?
If yes please, can some one send me the example code for that and treat me as a New Bee in this facebook integration?
Thanks in advance.
Have you tried out the various examples available in the api wiki?
http://code.google.com/p/facebook-java-api/wiki/Examples
Google also returns various tutorial pages:
http://programmaremobile.blogspot.com/2009/01/facebook-java-apieng.html
http://www.socialjava.com/socialjavastepbystep.html
You should try working through some of these and posting specific questions about problems you run into.
Note that the facebook-java-api project is now dead.
Use instead:
http://restfb.com/