Develop Reference

Logging in to website from android app to retrieve and parse information

I've been researching networking in android and have become familiar with the basic concepts. However, I've recently been working on an app intended for use by my school as a gradebook/assignment display of sorts. My school uses the following online gradebook: https://parents.mtsd.k12.nj.us/, and currently I'm trying to allow students to input their login credentials into my app which will then try to retrieve their respective information and display it in a gridview. However, I have never attempted to log in to a website from an android app before parsing data (I have gone through several similar questions but I am still a little confused)
Any help is appreciated!
[edit] After logging in I will use import.io to display gradebook data, website does not have api

If I understand You clearly u need to :
Connect to site using HttpURLConnection with user creditentials (do Post request to login page)
Obtain cookies from request (my favorite way is to parse - copy appropriate header field)
Use obtained cookies with HttpURLConnection.setRequestProperty("Cookie",obtainedCookies) and do another request(to user data page).
Get input stream
Parse stream
(to obtain Document "html" u can use jsoup example:
Document doc = JSoup.parse(stream, null, "");
Show data
GUIDELINES:
cookie how to
Most useful example
Caution:
-any http requests needs to be done outside main thread (ui thread) - u can use async task for this or intent service
EDIT on questions:
you ask whether to use Jsoup to handle connection?
my answer is NO!
Jsoup is a PARSER by using HttpURLConnection u gain full control over HTTP request so for example u can handle some specyfic exceptions or request propertiers which jsoup is not capable! from my experience after a while I start to disassemble the libraries and learn from them the basics and use parts of them in my code!

You could make a WebView with a custom HTML file which has a login and password field, and then have the HTML file run the same script as the website to log them in.

GAE: Incoming emails can not preserve format

I've set up my GAE/Java project to receiving emails and it works pretty fine excepting it can not preserve the incoming mail's format(e.g. bold, italic, font size, text color, bulleted list...), and the content type of incoming mails are always "text/plain", as a result from the end user's view the mail content huddled and unreadable.
For example I send a formatted mail from Gmail, when I receiving the mail in GAE all formats is tripped off and leaves a bulk of plain text.
Is there any way I can get incoming mail type as HTML so the format would be preserved?

While sending the mail through server. Set the body content type text/html.
.
.
.
htmlPart = new MimeBodyPart();
htmlPart.setContent("<b>html content</b>", "text/html");
This should work for you..

Looks like a duplicate of this question and answer
Moreover, I am copying a few excerpts from Google App Engine Documentation here which says:
The message contains a subject, a plaintext body, and an optional HTML body.
It can also contain file attachments, as well as a limited set of headers.
And I am guessing the content type should be text/html

Some questions related to implementation of image inside email signature?

i need to implement the email signature with image.As of now we only support the text in email signature which is already working.i need to provide the functionality
where i can insert the image inside mail signature. i can send the email to user within myapplication and also to user on external mail domain like gmail,yahoo etc. When
mail is sent to some user with in my application system, system makes entryt o DB and when receiver receives in inbox (which internally read the mail from db). Now if user
send the mail to external user on gmail it makes use of javax mail api . Similary i can receive the email from external mail domains(gmail,yahoo etc) Now i have
few questions based on tis requirement:-
1)Is there any standard for how the external mail domains like gmail send the image inside signature to another domains like (my application mail domain)?
Another point related to it gmail user can have two images ,one for signature and another image inside body. How will i determine which image belongs to
signature? Is there any defined property for that?
2)Also not able to make out what is the best/consistent approach to send(whether to internal application user or external mail domain user ) the email signature containing
image so that it renders correctly when user receives it?
what I had in my mind for point 2:- i earlier thought i can use solution suggested at How to display an image in jsp?. where
with tag <.img src="/getImage.action?imageId=123">, i can fetch the image from db in action class or servlet and return. But keeping in mind
once i send the mail to the user on gmail , he will not be able to access the servlet.So this approach does not seems to fit in requirement.
Then i came across the another great stackoverflow link base64 encoded images in email signatures where
solution by Tim Medora looked great but again the comment below the solution Gmail doesn't seem to support it again ended my Folks
really i think i should be done if mail domain like gmail,yahoo support the solution suggested by because in that case i can send image as base64 string instead
of image as attachment.
Folks would be really grateful if you can provide me some pointer/approach regarding both points 1 and 2

To include images in the email message, first you have to include the images as MIME attachments in the email. Each of these attachments must have a "Content-ID" header.
--f46d0444ea0d6991ba04b91c92e6
Content-Type: image/gif; name="theImage.gif"
Content-Transfer-Encoding: base64
Content-ID: <theImage#abcd>
[base64 string]
--f46d0444ea0d6991ba04b91c92e6--
2) Then, in the email message, include the Content-ID in the src attribute of the <img> tag.
<img src="cid:theImage#abcd" />

For Gmail to see the embedded image from byte array, I posted an answer on another similar question which is to use ByteArrayDataSource and embed it to the HtmlEmail. Here's the code snippet:
import javax.mail.util.ByteArrayDataSource;
import org.apache.commons.mail.ImageHtmlEmail;
...
ImageHtmlEmail email = new ImageHtmlEmail();
byte[] qrImageBytes = createQRCode(); // get your image byte array
ByteArrayDataSource qrImageDataSource = new ByteArrayDataSource(qrImageBytes, "image/png");
String contentId = email.embed(qrImageDataSource, "QR Image");

Logging in remote site then testing the form

There's a website www.runescape.com and I'm trying to find a way to login the site and automate entering 2-3 letter words (from a-z0-9) to find a unique list of available usernames. (https://secure.runescape.com/m=displaynames/name.ws)
This is the only page I know of where you can view usernames that are available or taken.
In the form where you change your name--you type in a name and it queries it for you with no form submit needed. In about a second or two, the page will display a lime green "Available" or a red "Not available" in text.
Is there a way in java where I can login to see this page, then enter in values, and read if username(s) are Available or Not available? Can someone help point me in the right direction to do this?
Much appreciation.

By viewing the source code of the web site, I can see that the form is sent via POST to https://secure.runescape.com/m=weblogin/login.ws. You will want to send an HTTP Request to that address with the associated username and password parameters respectively.
There's a Java HTTP Request tutorial on SO Here: Using java.net.URLConnection to fire and handle HTTP requests
After you've sent the HTTP Request to the page, an HTTP Response object should be returned to you that has the body of the entire web page that you're looking for. Most likely, you'll need to use RegEx to scan through the body of text and filter out the data you want e.g. <li>([a-z0-9]{1,20})<\/li>
Handling the HTTP Response object is also available on the link provided.

Is it a good practice to use asynchronous requests in this scenario?

I have a scenario that I have a button in JSP page which sends an email, the request is send to servlet asynchronously using jQuery Ajax and JSON, servlet searches in DB, if the user has an email, it returns the email address and sends an email to it, then forwards to the result page with success or fail of sending the email, but in a case that the user doesn't have an email, it returns false values using JSON to JSP and then a JSP form appears to the user to enter his email.
Is it good practice to use Ajax and I know that not each time there's a return value to the user or send request to servlet using get method which return a parameter in a case that the user doesn't have an email?

Using ajax is in practically all cases very good for User Experience. With ajax, the user will experience instant feedback without the need to face an annoying "flash of content" or a (partially) empty page because the whole HTML response needs to be generated/buffered by the server first. This is really a huge plus of using JS/ajax.
Using JSON is generally favorable above XML, HTML or even plain text. But there is no "best practice" with regard to the ajax data exchange format between client and server. Just pick whatever suits the requirement the best. JSON is perfectly fine for this case. jQuery understands it out-the-box and in Java you have choice of a plethora of easy-to-use JSON parsers.
However, when developing an ajax-enabled webapplication, you really need to take into account that the core functionality does not break when the client has JS disabled. This is called Unobtrusive JavaScript. Most of the searchbots, mobile browsers and textbased browsers don't use JS. You should try to use JS only for Progressive Enhancements. To test this yourself, in Firefox you can use for example the Web Developer Toolbar to easily enable/disable JS support. Run your website with JS disabled and observe if the core functionality is maintained as well.
The best way to achieve this is to start developing the website without any single line of JS code, even without a single onclick, onsubmit, onwhatever attribute on the HTML elements. Once you get the core functionality to work, then you can start adding JS in flavor of a script which executes during document ready and attachs functions to the HTML elements of interest (even here, you should not change the original HTML code!). Let the JS functions fire ajax requests on the same URL or maybe a different one, depending on the requirement. You can in the Servlet distinguish between an ajax and normal request as follows:
if ("XMLHttpRequest".equals(request.getHeader("X-Requested-With"))) {
// Handle ajax request. Return JSON response here.
} else {
// Handle normal request. Return normal HTML response here (by JSP).
}
See also:
Simple calculator in JSP - contains unobtrusive JSP/Servlet/jQuery example

Json is just a data-interchange format. Using Json or not has nothing to do with using asynchronous communication or not... You can do both communication types using Json (or XML, or serialized objects, it doesn't matter).
Now, in your problem, it looks like you just want to use Asynchronous communication to improve the user experience (it will not flick the user's browser). If that's the case, Asynchronous communication is the way to go!

I don't think you need ot use AJAX in this.
The main idea of the ajax is to render server response without postback and in your case you are redirecting page after you get some kind of result.
In my opinion you shoul choose on of these two ways.
1) Use AJAX, send data to servlet and then render response from server wether the mail is sent or not.
2) Submit your form to servlet and sent email and then redirect to jsp with the success/fail result.
Hope it helps.