Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I would like to create website which quizzes people on animal facts, I could do it manually, but I am wondering how I would go about automating the 'retrieval of facts' from the internet and then inputting it into the quizzes automaticallly. I'm thinking, I could build a web crawler/scraper that collects parts of information from popular sites, and then sends that data to the site, to make the quiz.
How would you go about solving this issue, and if there is a better way of achieving the same think?
PS: I am a beginner, and I might not be using the right terminology
You should take a look at Scrapy.
Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Here is a nice tutorial page from the documentation: Scrapy Tutorial
Look up Selenium. It is my favorite library for python. It can log into sites, click on certain elements, and you can get data very easily.
Use pip install selenium to download
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I would like to write various web browser games and apps to post to my own personal website, mostly for fun. I'm pretty familiar with Java and starting to get into C a little. As I understand it, I cant just run a common java application from a web browser but have to use the JApplet class or Applet class(or possibly others.) Can I use a JApplet to make fairly sophisticated games or is it really just for basic display images? I'm willing to learn something completely outside of Java.
I've been browsing the web but am a little overwhelmed in deciding how I should go about this. I would like to make games that are pretty simple graphically, but can remember things like a characters inventory.
Its really just a place to host different games and things I make. How should I go about this?
I hope this isn't too vague.
The only language you can guarantee within a web browser is JavaScript. You can write highly sophisticated games using JavaScript, so this shouldn't be a major problem - you can see an example written using Unreal Engine and JavaScript on YouTube.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I need to design a testing system which would not allow to use google. I mean that, theoretically, the user still can do it, but the purpose is to hinder this as much as possible.
I have an idea, but I don't know how good it is. I want the question in the database to be a simple text. And when we need to load it to the test, we convert it into a picture with a watermark. But the problem is that, theoretically, it still can be recognized. Google is able to convert the text on the image into the text and then search it as a text. Google is also able to recognize images despite some hue differences on them. Does anybody have proposals on that score? What would you do?
just use robots exclusion headers on the page to tell google not to store your page. or even robots.txt
As you say if it's human-readable, google will read it. Google has millions of slaves doing OCR for them (via Re-Captcha). but if you tell google to leave your site alone, it will.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Hi I want to create a small web application which will take inputs such as 80 and 120 and return the GCD of those numbers. I want to write the logic in Java. When the user enters 80 and 120 and clicks on calculate button, the values must be passed to the algorithm and return the answer to be displayed on the application again. How do I link the java algorithm to the html page in order to achieve this? Kindly suggest. Is there no other way of doing this other than creating a Java applet for this and deploying it on the application? Kindly help.
Use "Applets" if you want the Java code to run in the users browser and not on a remote server.
Applets are, however, a waning technology due to the numerous attacks that has gone through the JVM to avoid the security checks in the browser, so your users will most likely not see the experience you want them to. Additionally I believe that the ability for the Java program to interact with its surrounding page has been crippled again for security reasons.
Your best bet is to use Java for server side code only (which it is quite fine for - Google Application Engine is perhaps the easiest way to get started) and to use JavaScript for client side code.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am trying to scrape data from two websites. I want to build a generic web parser to scrape those website. I need a generic parser because i might need to parse additional websites in the future. I want to parse the websites using java and store specific data in MySQL. What are the steps I should take? Can anyone guide me in this?
I've done this same thing for scraping the web using java and building a parser is an absolute nightmare, there are so many more things to consider that you realize at the start.
Use jSoup. It is sooo easy, and leaves you time to enjoy the finer things in life.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
How can I create a site that users can manually make their own websites using my control panel in the site like Yola or Weebly that are free web developers. I think they use JSP but not sure. Can you help me that what kind of language they make those sites?
You could use any server-side language to develop a "site for making sites". PHP, Java (JSPs), Ruby/Rails, Python/Django... there are a lot of options. Some languages are better for certain tasks than others, but pretty much anything on the market would suffice for creating the website-making site.
As for "how to create" such a "site-building site":
You would need to supply users with templates, tools to edit those
templates, and ways to change the metadata about their sites.
You would also need to provide hosting for your users. You might come
in the form of subdomains, like jacksonssite.awesomewebsitemaker.com.
You would also need to make sure that your templates create pages
that are secure from "hackers" and the like.
You might also need to create tutorials / hire a support staff, to
help the technologically-illiterate use your tools.
And many more things as well.
It would be a complex task. If you are a beginner to web programming, undertaking a project like this would be far too ambitious.