Retrieving data from web site without api - java

I need to query (using java) this site https://www.conversion-tool.com/bpmdetector/ in order to obtain bpm of a song previously uploaded.
The site has no API ready to use, so I don't know how to retrieve the value I'm interested in.
Have you got any idea on how to proceed or any java libraries that can be useful for me?

Have a look at: https://jsoup.org/ It can parse an HTML page into a Java object which you can iterate/query over to get the value you want

Related

How to get data from XML feed URL in Java

There is a betting exchange website which offer their data in XML from the following link:
http://odds.smarkets.com/oddsfeed.xml
I would like to access this link to retrieve the latest data (in java). Previously I have had to download the (very large) file and add it to my project and get the data from there. What is the best way to achieve this without having to download the file every time I want to access the data?
I plan on storing the returned data into a database.
Thanks
Well this seems to be very tricky question .I would suggest you to create a simple web service application[Client/server architecture] to get the contents from this url. You can use REST to call this url. But what contents you need to read depends on the functionality that you want to achieve.You need to write your custom logic to read the data.Here in you will be acting as client and the url would be your service.
You can refer following link
https://community.atlassian.com/t5/Confluence-questions/Access-page-content-via-URL/qaq-p/163060

What should i use to create web services app?

As an application to get a job I need to make a web app. I'm only familiar with Java SE so here comes my concerns. I need to make web service where at the beginning there will be authentication window, then I need to show the JSON data (probably parse it and show) as table or as list with button near to choose one of the row from the table to get to the next page where there user can choose a materials and so on.
I have data in JSON on server I need to pull it from there, then I need to show data which looks like this /materialDetails?ID=x where x is ID (it's probably HTTP or URI). Should I use Java REST? If yes I need to create a site in XML and then put java data inside? There're only a few tutorials on the internet and I can't find any good(sometimes the problem is in server, sometimes with dependencies). I was looking for information also on youtube but except https://www.youtube.com/watch?v=X36Dud8cS4Y I cant find anything useful? Could someone explain me this to make it at least a lil bit easier? Or just lead me to pick a specific framework. Thanks in advance
You could create a Dynamic Web Project with Tomcat and a MySQL Database for starters. You could use RESTEasy to create a WebService that gets data from your Database.
I don't know what exactly is expected from you, but this might be a good start. "Making a web app" is a bit like saying "I need to develop a java program", it is a bit vague !
I don't know REST but I think your application can be implemented with this technologies: HTML and Servlets/JSPs.
I would write an authentication page in HTML (one form element with 2 inputs and a button) which would pass credentials to a Java Server Page or a Servlet (they're equivalent). There I would build the table (another HTML element) thus producing a new HTML page.
P.S.: you're using JSON as a format so there's no need to learn XML.

Google page index - java

does any one has idea how to get google page index in Java?
I was googleing since last 2-3 days but helpless, can any one refer me API for that or give some suggestion for how to do that
Lots of thanks in advance
For example if we search for facebook in google, we get around 22,980,000,000 results. So I want to fetch this number using JAVA
make a corresponding HTTP request from Java to Google, then parse the replied HTML code. There is a div with the ID resultStats. This div contains the number of results.
Not sure what your real requirement is, what kind of index do you want? Google export fairly a bit amount of APIs via RESTful service, some of them are packaged with JavaScript lib like Google MAP API. There are also Java client library for OAUTH authentication
The custom search API information could be found at http://code.google.com/apis/customsearch/v1/overview.html. A comprehensive list of google APIs could be accessed at https://code.google.com/apis/console

How to fill out form data on a website

I am looking to develop an app that will take login details from the user, go to a website, login, return values on the web page and then display them to the user on the phone.
Does java have this functionallity? Will I need to use javascript instead maybe? do these answers depend on the website that I am trying to access?
In my head I figure that I could just read in the paramaters as strings or chars, parse the webpage for the appropriate form and "paste" the appropriate value into the form "box". However, I have never attempted anything like this with coding so I am completely new to the idea and dont really know where to start. I tried googling around but any information that I found was either irrelevant or conflicting.
I'm not looking for the code to do it because I will not really learn anythig from that but a finger in the right direction would be great. I really do want to try get better at programming so that's why I've started to give myself these little side projects
Any help that can be offered would be great
Ian,
You can try using http-client (http://hc.apache.org/httpclient-3.x/) lib from apache. It lets to pro grammatically access a website (from a Java code). You will need to do the following things
Use the http-client lib to POST the data to the web site.
Receive the html response.
Use some html parser or xpath to retrieve the values from the response html.
You would need a script which accesses the webpage and enters the data, but in my opinion this is illegal. Because you are accessing a secured area and are able to look into sensitive data. Also accessing the page via a script is "botting" - most pages have safety precautions to prevent the execution of scripts, because most of them are harmful.
In my opinion there is no legal and easy solution to this.

java html parser doesnt read all page

I'm parsing html pages to get specific information, but there are some pages that I cant get all the information displayed on the web page, for example in this page
I cant get the reviews information.
By the way, if you see the source code of the page there are very much empty lines, and the reviews information dont appear.
Do you know why?
Some library to read this type of pages?
Thanks
I'm willing to bet they are using some sort of javascript to load in the review information. In order to access that information, you are going to need to somehow either mimic the request or evaluate the javascript and then parse the resulting page. I would suggest examining their javascript and mimicking the request they use to download the review information as that will be much easier than attempting to evaluate the javascript in your code.

Categories

Resources