Develop Reference

Logging in to website from android app to retrieve and parse information

I've been researching networking in android and have become familiar with the basic concepts. However, I've recently been working on an app intended for use by my school as a gradebook/assignment display of sorts. My school uses the following online gradebook: https://parents.mtsd.k12.nj.us/, and currently I'm trying to allow students to input their login credentials into my app which will then try to retrieve their respective information and display it in a gridview. However, I have never attempted to log in to a website from an android app before parsing data (I have gone through several similar questions but I am still a little confused)
Any help is appreciated!
[edit] After logging in I will use import.io to display gradebook data, website does not have api

If I understand You clearly u need to :
Connect to site using HttpURLConnection with user creditentials (do Post request to login page)
Obtain cookies from request (my favorite way is to parse - copy appropriate header field)
Use obtained cookies with HttpURLConnection.setRequestProperty("Cookie",obtainedCookies) and do another request(to user data page).
Get input stream
Parse stream
(to obtain Document "html" u can use jsoup example:
Document doc = JSoup.parse(stream, null, "");
Show data
GUIDELINES:
cookie how to
Most useful example
Caution:
-any http requests needs to be done outside main thread (ui thread) - u can use async task for this or intent service
EDIT on questions:
you ask whether to use Jsoup to handle connection?
my answer is NO!
Jsoup is a PARSER by using HttpURLConnection u gain full control over HTTP request so for example u can handle some specyfic exceptions or request propertiers which jsoup is not capable! from my experience after a while I start to disassemble the libraries and learn from them the basics and use parts of them in my code!

You could make a WebView with a custom HTML file which has a login and password field, and then have the HTML file run the same script as the website to log them in.

Real-time Process Progress Logs for the User

Well, I'm Trying to Make a Data Importing Module. From the module, the user choose the .txt File with Data and then click the upload button. I want to make a Textarea or textbox (My project is a Java EE WebApp) where the webapp shows the real-progress of the upload proccess with Descriptive Messages.
I'm thinking (And i've searched) about Multiple Ajax Requests, and, Multiple Ajax Responses with one Request (The last one is not valid, as i read), but, i'm confused about the usage of AJAX in this case. It is Valid the user hit "Upload", and then, i call an AJAX Request that returns the text with the progress of the actual registry imported?
I'm thinking to use:
jQuery 1.6.2
GSon (For ajax)
Any suggestion would be appreciated

I would recommend using JBoss RichFaces 'poll' mechanism for that, or just a simple jquery script on the client side:
Ajax Poll Example with RichFaces: http://richfaces-showcase.appspot.com/richfaces/component-sample.jsf?demo=poll&skin=blueSky
JQuery (loads of examples on the web):
http://net.tutsplus.com/tutorials/javascript-ajax/creating-a-dynamic-poll-with-jquery-and-php/
jQuery AJAX polling for JSON response, handling based on AJAX result or JSON content

How about using a iframe that handles the upload form? This way it would not require the browser to update (by AJAX calls) the contents of a page that "we're already leaving". The iframe could be styled so that it's indistinguisable from other content.
AJAX-calls to a some method that keeps an eye on to some progress-variable (lets say a double that indicates percentage) is perfectly valid. Below is a barebones pseudo-example.
!PSEUDO!
double progress = 0.0d
void upload(request, response) {
// updates progress real-time
}
void ajaxProgress(request, response) {
// set progress to response
}

You may want to consider all the traffic back and forth showing real time processing information of uploaded files.

How to programmatically send a HTTP request with parameters? [duplicate]

This question already has answers here:
How to use java.net.URLConnection to fire and handle HTTP requests
(12 answers)
Closed 7 years ago.
If I use a browser to send information to the server (for example using a log-in, password page), I just fill the user text-box and the password text-box and clicking on the log-in button.
I would like to send this information but without having to use the browser. I would like to 'fill' the text-boxes but without having to do it manually in the browser. May be using a Servlet.
My question is: How to send information in text-boxes, for example, to a website, doing it from a Servlet?

why not just make a call to the URL from Java using a URL like http://your.domain.name/your/servlet/path?userFieldName=THE_VALUE_YOU_WANT_TO_PASS&passwdFieldName=PASSWORD
The servlet will feel like the values are coming from those boxes.
Or you may want to dive into Apache HTTP Client to mimick a request sent from an client.
uh..oh.. are you doing functional testing? Why not look into JMeter?
Updates as per comment
You need to know what actually form submission does? It basically forms a query string composed of Key-Values (KV) pair.
So, if you have a a text field named tfield where user has typed some text, and there is a drop down named, ddfield where user has selected optionX which has value optionX-Val. And this form gets submitted to a URL, http://my.domain.name/my/servlet -- the browser will send a request which will look like
http://my.domain.name/my/servlet?tfield=some%20text&ddfield=optionX-Val
If you want to mimic form submission, you will have to manually create a URL that has a request string containing all the fields and their values as FIELD_NAME=FIELDVALUE ordered pair separated by ampersand (&)
ah, great idea. If you use Firebug (a Firefox extension), open the NET panel in Firebug, make a manual submission of the form that you wanted to mimic. See what request is posted when you submitted the form. It will have exact URL format that you are after. Copy this URL, replace the values and make fake submissions as much as you want.
Hope this helps.

It is not clear to me what you really up to. I assume that the servlet will be the one who will send the data. Here some examples.
Using setAttribute then Forward the request
//On your servlet
request.setAttibute('user', 'admin');
request.setAttribute('password', '123');
getServletContext().getRequestDispatcher("page.jsp").forward(request, response);
//On your jsp page get the value using EL
<span>${user}</span>
Using session
//On your servlet
HttpSession session = request.getSession(true);
session.setAttribute('user', 'admin');
session.setAttribute('password', '123');
getServletContext().getRequestDispatcher("page.jsp").forward(request, response);
//On your jsp page get the value using EL
<span>${user}</span>
The above example is intended to work within the web application. To send information to another web application, which expecting a request. See sample below.
//On your jsp or servlet, you can also do the same within web application
request.sendRedirect('http://example.com?user=admin&password=123');
//on your jsp #example.com
<span>${param.user}</span>
If this is not what you mean, adding more details will be a help.

a servlet takes care of the other end: it's basically a handler for http requests that lives inside a servlet container. If I understand you correctly, you're wanting to send an http request. You can do that using command-line tools like curl, or if you want to stay within java land, you could try this example on exampledepot. Use your favourite search engine to search for more examples, e.g. with search terms such as "sending GET requests through a url".
In your situation, where you need to send information for username and password, you would need to look at the html and find the url for the form element's action attribute. Then you need to find the names of the username and password fields. Using these names as url parameters, you can construct a GET request that mimics sending a form.
NOTE: usually storing a password in plain text in code and/or sending it in plain text to a website is not a good thing to do.

Just in case anyone is interested, there is a plugin for Firefox called Tamper data. With it you can stop the sending of and http request and modify it. It will show you the "url" you need for sending the params, the values they currently have, and their name. You can check it out here. After that you can use a request.sendRedirect('url you got from Tamper Data');

Is it a good practice to use asynchronous requests in this scenario?

I have a scenario that I have a button in JSP page which sends an email, the request is send to servlet asynchronously using jQuery Ajax and JSON, servlet searches in DB, if the user has an email, it returns the email address and sends an email to it, then forwards to the result page with success or fail of sending the email, but in a case that the user doesn't have an email, it returns false values using JSON to JSP and then a JSP form appears to the user to enter his email.
Is it good practice to use Ajax and I know that not each time there's a return value to the user or send request to servlet using get method which return a parameter in a case that the user doesn't have an email?

Using ajax is in practically all cases very good for User Experience. With ajax, the user will experience instant feedback without the need to face an annoying "flash of content" or a (partially) empty page because the whole HTML response needs to be generated/buffered by the server first. This is really a huge plus of using JS/ajax.
Using JSON is generally favorable above XML, HTML or even plain text. But there is no "best practice" with regard to the ajax data exchange format between client and server. Just pick whatever suits the requirement the best. JSON is perfectly fine for this case. jQuery understands it out-the-box and in Java you have choice of a plethora of easy-to-use JSON parsers.
However, when developing an ajax-enabled webapplication, you really need to take into account that the core functionality does not break when the client has JS disabled. This is called Unobtrusive JavaScript. Most of the searchbots, mobile browsers and textbased browsers don't use JS. You should try to use JS only for Progressive Enhancements. To test this yourself, in Firefox you can use for example the Web Developer Toolbar to easily enable/disable JS support. Run your website with JS disabled and observe if the core functionality is maintained as well.
The best way to achieve this is to start developing the website without any single line of JS code, even without a single onclick, onsubmit, onwhatever attribute on the HTML elements. Once you get the core functionality to work, then you can start adding JS in flavor of a script which executes during document ready and attachs functions to the HTML elements of interest (even here, you should not change the original HTML code!). Let the JS functions fire ajax requests on the same URL or maybe a different one, depending on the requirement. You can in the Servlet distinguish between an ajax and normal request as follows:
if ("XMLHttpRequest".equals(request.getHeader("X-Requested-With"))) {
// Handle ajax request. Return JSON response here.
} else {
// Handle normal request. Return normal HTML response here (by JSP).
}
See also:
Simple calculator in JSP - contains unobtrusive JSP/Servlet/jQuery example

Json is just a data-interchange format. Using Json or not has nothing to do with using asynchronous communication or not... You can do both communication types using Json (or XML, or serialized objects, it doesn't matter).
Now, in your problem, it looks like you just want to use Asynchronous communication to improve the user experience (it will not flick the user's browser). If that's the case, Asynchronous communication is the way to go!

I don't think you need ot use AJAX in this.
The main idea of the ajax is to render server response without postback and in your case you are redirecting page after you get some kind of result.
In my opinion you shoul choose on of these two ways.
1) Use AJAX, send data to servlet and then render response from server wether the mail is sent or not.
2) Submit your form to servlet and sent email and then redirect to jsp with the success/fail result.
Hope it helps.

Web service for retrieving chunks of data

I'm planning to develop a webservice, and I like to try the RESTful architecture. The issue is that I don't know if the service is adequate for it, or it is better to use SOAP.
The service is about downloading some data from the server to a device on the local computer. The data will be split into chunks. The service will be run with an ad-hoc client at the local machine that will manage the device the file is gonna be stored in.
I was thinking on having something like:
/files/{id} --> will inform about the details of the file
/files--> list all the files
The problem is for the action. In rest only GET, POST and (PUT DELETE) are defined. But I want to have something like download. My idea, although not fully restful is to create:
/files/{id}/download
This will return something like
{ "chunk" : "base64 string with chunk data"
"next" : "http://XXX/file/id/download?chunk=1
}
When next is empty the whole set of chunks would be downloaded.
What do you think? Is it ok to do it this way or would it be better the traditional way using SOAP and defining functions like getFiles(), getFileChunk(chunkNo, file)?
Any comment is really appreciated.
See you

If using REST, you don't need to define your own "chunking" protocol as the HTTP headers Content-Length, Content-Range and Transfer-Encoding are all used for sending chunked data.
See the RFC for HTTP header fields

As John already mentioned you might want to separate between your file resources and the file resource metadata (any information about your file). Additionally a more RESTful way to access your chunks could look like this:
http://url/files/{id}/chunks
{
"complete" : false,
"chunks": [
"http://url/files/<fileid>/chunks/1",
"http://url/files/<fileid>/chunks/2",
"http://url/files/<fileid>/chunks/3",
]
}
Basically, here, you return a list of RESTFUL URIs to all your file chunks and the information if all chunks of the file are already complete. I don't see that SOAP might have any advantage there since you would define the same methods (getFile and getChunks) that are already covered by the REST verb GET.

It sounds like you really have two different resources: file-metadatas and files. What about something like:
/file/{id} // GET: Retrieve this file's data.
/file-metadata/{id} // GET: Metadata about a particular file. Contains link to file:
// {
// ...
// data: "http://.../file/156", // Where to find file's data.
// }
/file-metadata // GET: List metadata for all files.