java - parsing an aspx website - post parameters - java

I have my client's e-shop, which is created by another company. I want to parse all the products and put them in an xml. I know how to get to the first page of each "brand" but I have difficulties passing the argument to change the page for the paginated results.
This is the e-shop "http://www.gialia.net.gr/ProductCatalog/20/CAR.aspx" that points to one brand.
When I user tamper-data on firefox I see that when you want to press the second-page of the results is posts the :
"__EVENTTARGET=ctl00%24wpmMain%24wp131820866%24wp512420601%24dpgTop%24ctl01%24ctl01"
the last string: "ct101" means go to page 2, If I change it to ct102 it goes to page 3 etc.
BUT i'm trying to create it as a GET request so I can create these parameters dynamically in my Java code and parse each responce. But when I create the url as:
http://www.gialia.net.gr/ProductCatalog/20/CAR.aspx?__EVENTTARGET=ctl00$wpmMain$wp131820866$wp512420601$dpgTop$ctl01$ctl02
I get no results.
Can someone please take a look and give me some suggestions?

The site you give us here is very poor in design concerning the search engines (SEO), and so the parse of the page one by one is too difficult.
To change page is make post back, and with javascript only. So you must do the same to move to the next page of the catalog, you need to make a full post back of the page with all the parameters.
Now, the page is so bad designed that the programmer have disable the __EVENTVALIDATION of the controls probably because he not let him do wrong things, so when you can tamper the data, but still you need to make post back. By simple type on the url one only parametre the code behind did not understand that is post back. You need to send and at least the Viewstate and the rest hidden parameters.
But isn't more easy to just get from your client access direct to the database and reads them from there ?

Related

Java web development: transfer control from one servlet to another while passing the request object [duplicate]

This question already has answers here:
Design Patterns web based applications [closed]
(5 answers)
Closed 6 years ago.
EDIT: I have posted a somewhat shorter and revised question here: Java web development: transfer control from one servlet to another while passing the request object (Version 2)
As more or less a beginner at Java web development, I’m unsure about how I should structure the flow between servlets/pages when a form is submitted (POST). It’s an elementary issue, I suspect this may be an easy question to answer for the experts. (Still, my book and some googling didn’t deliver a clear answer.) My question is a bit long, and that's because I want to make it clear where I'm coming from. Thanks for you patience.
Let’s say we have two servlets A en B, with each having its ‘own’ .jsp-page; let’s call those pages a.jsp and b.jsp respectively. Now as long as there are no forms on either page (i.e., no POST method used), it’s clear how things should go. That is, before any .jsp-page is shown, the corresponding servlet is activated, doing some preparation for the .jsp-page by setting the relevant data elements (most notably, as attributes of the request object) that the .jsp-page needs, then forwarding the request object (etc.) to the .jsp-page, which then actually displays the page with the data. So for example, a link on page a.jsp may link to the servlet B, and on clicking that link a GET-request for servlet B is triggered, which then does some preparation (setting some request attributes), before forwarding to its ‘own’ .jsp-page (i.e. b.jsp).
But now let’s assume that page a.jsp displays a form with a submit button, method=”POST” and action=”B”. Then yes, servlet B is activated, and this servlet has to determine whether the data entered by the user is valid. If the data is in fact valid, we can simply forward to b.jsp, no problem there. But what if the data is NOT valid?
In that case, we obviously want to show a.jsp (the form page) again, with the data that the user entered the first time still present. One way to achieve this, is to simply have servlet B forward to a.jsp (thus bypassing servlet A). However, there is a big problem with that: the URL shown to the user, in the address bar, will still read “……/B”. So the user will see the correct page (i.e., a.jsp, containing the form), but with the wrong URL (/B). So for example, if we take “Register” and “ThanksForRegistering” instead of “A” and “B”, the user will see register.jsp – but with URL “……/ThanksForRegistering”! Not good.
And calling ‘include()’ instead of ‘forward()’ on the request-dispatcher doesn’t seem to work either. If we do that, not only does it result in a GET-request (as opposed to the POST-request we want), but we actually lose the whole (original) request-object with its attributes (which we need, after all, to re-populate the form). At least, that’s what my own experimentation seems to show. So using ‘include()’ doesn’t seem like a viable option at all.
Another obvious idea is to have "action=A" (instead of "action=B") for the submit. Then the servlet A itself can handle the validation, and if validation fails it can simply forward to a.jsp again, no problem. BUT then what if validation succeeds? Then we want to show the follow-up page b.jsp, but that page may well need the attributes from the original request-object (from the form-submit) again; for example, to have the user check that his entered data was in fact all correct. So basically we have the same problem as before, but with the roles of A and B (and their respective .jsp-pages) reversed. So this doesn't seem like a real solution either.
And I don’t see any other alternatives.
So basically, I’d simply like to be able have one servlet give control back to another servlet, but with the request object being passed from the former to the latter servlet. Or, if that’s not possible, I’d want to be able to forward from servlet B to a.jsp directly, but with the correct URL shown to the user. Or any other way to accomplish what I want.
Many thanks.
I think that the assumption that there has to be one page per servlet is causing the problem here....have one servlet which based on input redirects,forwards or includes a particular page....you dont really need to always invoke a different servlet for a page.....you can have a single front controller with a view resolver the combination of which will redirect or forward to a page.
You can use filters to achieve the same thing or think of setting attributes in HttpSession if validation is successful and retrieve the data in all the pages whenever it is required.
session.setAttribute("object", object);
I hope this is what you are looking for.

Post to a aspx site, and return and view xml in android app

im trying to make a simple android app. I want to post a request to this aspx site.I have a list the users picks a choice from. When the user chooce a listitem, I want to have an if statement that inputs the correct post variables to the site. I assume I have to use some kind of http post package. When the post is sent I want to return the xml that the site gives the http_post and show it to the user.
How would I go about this? I cant seem to find any way to show the result to the user. And I'm unsure if this is the best approach.
Is there a simpler way? Originally i thought of something like offer the user a simple WebView with a unique link to each item in the form list. But I'm unable to create independent direct links due to that annyoing .aspx server-side handling.
Could anyone help me out? :)

How can I send a newsletter with xPages content?

I have some content displayed using computed fields inside a repeat in my xpage.
I now need to be able to send out a newsletter (by email) every week with the content of this repeat. The content can be both plain text and html
My site is also translated into different languages so I need the code to be able to specify the language and return the content in that language.
I am thinking about creating a scheduled lotusscript or java agent that somehow read the content of the repeat. is this possible? if so, some sample code to get me started would be great
edit: the content is only available to logged in users
thanks
Thomas
Use a java agent, and instead of going to the content natively, do a web page open and open the page as if in a browser, then process the result. (you could make a special version of the web page that hides all extraneous content as well if you wanted)
How is the data for the repeat evaluated? Can it be translated in to a lotusscript database.search?
If so then it would be best to forget about the actual xPage and concentrate on working out how to get the same data via LotusScript and then write your scheduled agent to loop through the document collection and generate the email that way.
Looking to the Xpage would generate a lot of extra work, you need to be authenticated as the user ( if the data in the repeat is different from one user to the next ) to get the exact same data that this particular user would see and then you have to parse the page to extract the data.
If you have a complicated enough newsletter that you want to do an Xpage and not build the html yourself in the agent, what you could do is build a single xpage that changes what's rendered based on a special query string, then in your agent get the html from a URLConnection and pass the html into the body of your email.
You could build the URL based on a view that shows documents with today's date.
I would solve this by giving the user a teaser on what to read and give them a link to the full content.
You should check out Weihang Chens (my colleague) article about rendering an xPage as Mime and sending it as a mail.
http://www.bleedyellow.com/blogs/weihang/entry/render_a_xpages_programmtically_and_send_it_as_a_mail?lang=en_us
We got this working in house and it is very convenient.
He describes 3 different approaches to the problem.

How to retrieve query parameter in Wicket

I am using wicket application to initiate my Java method in my code. I have the url as follows
http://localhost:8080/web/resources/org.apache.wicket.Application/StartMethod
I want to add an extra option at the end client=cutomser1
http://localhost:8080/web/resources/org.apache.wicket.Application/StartMethod?client=customer1
As I want these use the same code but for different customers
So when I run this URL from command promte it should fetch the Customer1 into my Java code
in this way..
String client = customer1
How can I implement this in Java. I mean I want to fetch this String name customer1
Wicket is strong in data binding and you should take advantage of that.
If you display in the browser data from Customer1, the Wicket knows that when you do requests from that page.
The translation of request parameters to variables happens behind the scene.
That is disconcerting at first but really powerful when used as intended.
I would suggest you select a good tutorial from the answer to this question and go through a couple of examples to see how things are done. This may take an afternoon but this will pay back for itself before the week is out.
I think I am a bit too late, but still I will say that you can use PageParameters in your page and you should make the page bookmarkable.

Java Web Application

I am interested in creating a simple web application that will take in user input, convert it to an XML file and send the file to a database.
Coding wise I feel I am okay, it is just the general setup and what implementation to use I am a bit unsure of.
At the moment I have a JSP page containing a form, the user fills out the form and on submit a POST method is sent to a servlet, in the servlet doPost() method the servlet is instantiating a java object and passing it the user inputted data. The java object then writes that data to an XML file and sends it to the database via REST.
All I would be interested to know is if this the standard/optimal way of creating such a web application.
Any and all feedback is appreciated.
Thanks
For a "simple webapplication" this high level approach looks fine in general. However, if you want more critical feedback, you'd need to give more details about the low-level approach. It may for example happen that it isn't memory efficient and thus may break when the webapp is been used by over 10 users concurrently, just to give an example.
I only question the choice for the GET method. You'd normally only use it to retrieve data (SELECT), not to create/alter data (INSERT/UPDATE/DELETE). For that you'd normally use POST, so that no one can execute it "accidently" by just clicking a (bookmarked) link. Changing GET to POST isn't that hard, add method="post" to the <form> element and rename doGet() to doPost().

Categories

Resources