I need a java code to use wikipedia api. I would like to get the tags of a specific value on wikipedia (the tags that appear at the end of the page). I know that the api call I need is
"http://en.wikipedia.org/w/api.php?action=query&format=json&titles=Albert%20Einstein&prop=categories"
(for the example of Albert Einstein)
But I find it difficult to understand from the MediaWiki API documentation page if I need a token and hash, or to log in or not. I know that sometimes it is not necessary to log in.
Right now my code is:
JSONArray wikiResult=null;
String url1= "http://en.wikipedia.org/w/api.php?action=query&format=json&titles=Albert%20Einstein&prop=categories";
wikiResult=ApiCall(url1);
where ApiCall is a function that sends the call, and it works fine, I have checked it.
the JSONArray returned is empty.
Can somebody please tell me what I'm doing wrong?
You don't need to log in or get token for prop=categories.
The actual error is probably explained in the response. It's possible that you're getting an empty result, because the response is not JSON (that happens with some errors).
If I were to guess, I think you're not setting the User-Agent header in your request, which is required by Wikimedia wikis.
Related
I have my client's e-shop, which is created by another company. I want to parse all the products and put them in an xml. I know how to get to the first page of each "brand" but I have difficulties passing the argument to change the page for the paginated results.
This is the e-shop "http://www.gialia.net.gr/ProductCatalog/20/CAR.aspx" that points to one brand.
When I user tamper-data on firefox I see that when you want to press the second-page of the results is posts the :
"__EVENTTARGET=ctl00%24wpmMain%24wp131820866%24wp512420601%24dpgTop%24ctl01%24ctl01"
the last string: "ct101" means go to page 2, If I change it to ct102 it goes to page 3 etc.
BUT i'm trying to create it as a GET request so I can create these parameters dynamically in my Java code and parse each responce. But when I create the url as:
http://www.gialia.net.gr/ProductCatalog/20/CAR.aspx?__EVENTTARGET=ctl00$wpmMain$wp131820866$wp512420601$dpgTop$ctl01$ctl02
I get no results.
Can someone please take a look and give me some suggestions?
The site you give us here is very poor in design concerning the search engines (SEO), and so the parse of the page one by one is too difficult.
To change page is make post back, and with javascript only. So you must do the same to move to the next page of the catalog, you need to make a full post back of the page with all the parameters.
Now, the page is so bad designed that the programmer have disable the __EVENTVALIDATION of the controls probably because he not let him do wrong things, so when you can tamper the data, but still you need to make post back. By simple type on the url one only parametre the code behind did not understand that is post back. You need to send and at least the Viewstate and the rest hidden parameters.
But isn't more easy to just get from your client access direct to the database and reads them from there ?
I am trying to use the google-voice-java API to read texts from a google voice account. I cannot find much documentation on using it other then the code.google.com page. I just need a little help on how to correctly use getSMS();
The google-voice-java API will simply return back raw HTML of your Google Voice data. Thus, you need to parse through the HTML returned by getSMS();, and do with the data as you see fit.
If you'd like to just see what the data returned looks like at a console window, do the following
Voice voice = new Voice(userName, password);
String sms = voice.getSMS();
System.out.println(sms);
Based on that, you can see what the HTML looks like and how to parse through it.
Edit: Probably worth noting that you need to include the additional dependency jars to your build path; that should be more explicitly clear on their GettingStarted!
Edit 2: Based on your comment below, there are the getUnreadSMS() and markAsRead(msgID), but the former appears to return read SMS messages along with the unread ones.
If you notice at the top of the response XML returned by the getSMS() method (and majority of the other API methods), there is JSON data in the <json> element, which seems to have all the necessary information you need (including a isRead variable to indicate if a SMS is marked read, or not).
Pull down a response from the getSMS() method and use this online JSON viewer to better inspect the data in the <json> element, so you have an idea of what's in there. When you paste in the JSON data into the online viewer, omit the leading <![CDATA and trailing ]> inside the <json> element.
I would just setup a JSON parser (maybe even use GSON) and create SMS model objects based on the JSON data, ignoring the HTML completely; for unread messages, you want JSON objects with the isRead field set to false, obviously. You could then use the id field to pass into the markAsRead(msgID) method, to mark them read (I just tested this method and it works).
we are trying to add a simple test using JMeter in a JSF Application. We followed the instructions in:
http://jmeter.apache.org/usermanual/build-adv-web-test-plan.html
It has a simple login page with user name and password and a submit button. You can see from the screenshots that we used a proxy. With the settings in the screenshot we are getting HTTP 500 Error. I am not sure if I placed the question in a right way.. Please ask if you need any clarification.
The error code is:
EDIT:
I think this is going to be the longest question of SO. But images are better than words sometimes. Anyway, what we have done is to sent the data that is equivalent to what we see in the firebug. But still getting 500 error. You can see in the attachments Tomcat log.
HTTP 5xx codes are related to server or application errors. Search log files first.
Your script don't need a "User Defined Variables" component because there's no variable expression that really need to be evaluated per thread/user.
The "Regular Expression Extractor" component suffice to extract the JSF ViewState value.
I suggest you to delete the last part of your expression, " />", and change the regular expression grouping (.+?) to (\w+?) 'cause it will evaluate to a few matches (probably only 2). Change the value of "Match No." field to 1 (no need to use random if all values matched are identical).
I didn't understand why you used both "XPath Extractor" and "Regular Expression Extractor" components to extract the same value. I prefer to use the last one when leading with html. XPath is better when treating with well-formed xml strings/files.
To capture a script from scratch, I suggest you to add a "HTTP Proxy Server" inside Workbench, configure it, start it, configure a browser to use this proxy and navigate those pages using the browser. This way you'll capture all requests made and request headers used by the browser you choose. After this, remove unnecessary requests and change query parameters, like javax.faces.ViewState, to the corresponding variables.
Consider using extractors (Pos-Processors) inside an HTTP Sampler prior to the one that will use the variable in Parameter Values. Ex.: if /EBS request comes first and /EBS/login.xhtml request have a javax.faces.ViewState parameter then, probably, /EBS response will contain a hidden input with the javax.faces.ViewState value.
This is a common make up of JSF application test scripts I use. Providing more information about the cause of the HTTP 500 error should clarify the way to a better solution.
On the Regular Expression Extractor for jsfViewState, add (?s) to the start of the regular expression. So you have:
(?s)<input type="hidden" name="javax\.faces\.ViewState" id="javax\.faces\.ViewState" value="(.+?)" />
This allows the (.+?) to span line break characters.
Your regular expression extractor is in the wrong place. You cannot extract a value from the response to a request and then send it with the same request. The only way to achieve this is to use a time machine, but these don't exist yet and even if they did, it probably wouldn't work.
Typically you get a viewstate in the response to a GET and then you later need it in the POST of the same page. So, put the regular expression extractor as a child of the GET call where the login.xhtml page is first called (as a GET). If your recording does not include this GET call then either add it manually or examine the responses of previous calls before your login POST to find it, eg. maybe the GET homepage.xhtml (or similar) will include it.
I want to read a JSON list from a webservice with Java. The webservice returns a list of authors from luxemburg, e.g. sorted by the year. That's the web-site:
http://www.autorenlexikon.lu/page/periods/1919-1945/1/1/DEU/index.html
So far, I know that I can receive a JSON document with a request like this:
http://www.autorenlexikon.lu/mmp/json.document_list/DEU/0?search_since=1919&search_until=1945
But I only get the first 20 entries. How can I get the next 20 entries? I think the solution is in the JavaScript-code of the web-site, but I am pretty new in JavaScript (also in JSON).
EDIT:
There isn't any official API.
I have already tried:
http://www.autorenlexikon.lu/mmp/json.document_list/DEU/0?pageSize=1000&search_since=1919&search_until=1945
http://www.autorenlexikon.lu/mmp/json.document_list/DEU/0?page_Size=1000&search_since=1919&search_until=1945
...and many more. Who does the JavaScript-code receive all entries? Couldn't I copy this mechanism?
You should check their API and look for a parameter that let's you define the page or the range of results you want to get.
Edit Seems like you'd have to make a POST request and add the start index as well as the page size as post parameters. For more information see #matthijs koevoets' answer.
It depends on how the Webservice has been coded. Nothing to do with JSON specifically. From the results you can see it says
"pageSize":20,
You just have to figure out how to call the Web service with a page size. It may not allow you to query it with a different page size. That's up to the Web service API coded by their developers
their service seems to accept POST parameters only: sort=year&dir=asc&startIndex=0&results=100
I am writing a RESTful web service using Java which should be able to take as (POST) input an unknown number of parameter name, value pairs. I am thinking what is the best way to achieve this.
The POST input will be JSON with the expected format being:
{"input" :
{ "x" = 1, "y"="Hello, "z"="1.2.3.4",... }
}
The parameter values are expected to be a mix of int/long/float/string etc. The expected behavior is to save this input to the DB and provide it back on request. The tricky part is with the parameter names being unknown, I cannot write getter and setter methods. Is there a simple way for handling this?
Any pointers in the right direction will be very much appreciated. Can I read this as a List? Do I need to write Custom MessageBodyReaders? Is there some other way of achieving this? The only option I can think of is to have these sent as a file and parse it but am hoping that there are simpler/cleaner solutions. Thank you for any pointers...
If all you need to do is save it and make it available on request, why parse it at all? Simply write the json as a string to your database, and return it as a string while setting the response type to "application/json". The one caveat being you're going to have to have some way to identify which json string you want and associate a unique id that you're going to be able to determine when you're requesting the specific json string in the future.
Something worth noting here would be that when you don't parse something, you don't actually know what it is. Even though a lot of modern libraries/frameworks sanitize against sql injection/xss already, it would be worth it to verify that this is being done in your server if you don't already know for sure.
That being said, if you're still feeling as though you need to parse the json, I would look into gson: http://code.google.com/p/google-gson/