Develop Reference

How to capture all Network traffics(http\https) of a Web Application using Java in Eclipse IDE?

I want to record a Web Application traffic using Java code in the back-end and replay it after recording.
Is there any way to record Web Application traffic like we do in Load-runner using Java code?

Although it doesn't automate everything or is not specific to Java, you can use Wireshark to retrieve HTTP/HTTPS requests and export them ( https://www.wireshark.org/docs/wsug_html_chunked/ChIOExportSection.html ). Some other tools such as PlayCap allow to directly playback the wireshark capture.
Or you can write some Java code that mimics the requests and even load the wireshark capture file to send logged requests with any HTTP-able library.
You can also use directly pcap to retrieve packages (or the jpcap variant, written in Java, that may have some reusable code for your use-case), and try the similar approach to playback the capture.

Communication between javascript and its hosting java or c++ application

I'm trying to find out if there is a standard or recommended way to communicate from javascript to the application which embeds a browser widget, and vice versa. The hosting application may be written in either java or c++ and may run on Windows and Unix platforms, but the javascript would be shared across both clients.
So far I've read about:
window.external (This seems to be IE specific, so it wouldn't work on Unix.)
LiveConnect (This seems to be java and mozilla specific, so it wouldn't work for IE or c++ based applications.)
SWT's Browser widget has some of this capability, but this would be a java-only solution..
What other options are out there?
Thanks!
Shyam

We have a VB6 application that hosts Microsoft's WebBrowser object (IE). We've used a simple URL intercept mechanism to facilitate communication between the browser and the hosting application. Since the browser control has a before navigate interface, we can pull out the URL and examine it for commands and either cancel the navigation event (since it was meant for the hosting app) or let it pass through (since it is a normal URL).
We used something like app://commandName?arg1=val&arg2=val in our Javascript or HTML link tags.
Then in the BeforeNavigate event from the browser, we check the url for app:// if we get that, we know the browser is sending the parent application a message.
Simple but effective (for our needs anyway).
EDIT
Should also mention that most embedded browsers also have mechanisms to manipulate the DOM. That in mind you should be able to extract information (HTML nodes) and inject information at will.

JavaScript has the XMLHttpRequest API that makes it possible to send data to, and retrieve data from a server. The use of this API with messages formatted in XML or JSON is designated AJAX.
AJAX can be used to implement the example you gave, of a tree node in the HTML/javascript that retrieves the list of children from the server when it is expanded.
Note that when using AJAX, the server may be written in any language (C, Java, Python, Ruby, etc).
I suggest you to read about AJAX. After you get a good understanding of AJAX you can read a little bit about web services. web service is a method of communication of 2 applications developed in arbitrary programming languages through the WEB.

Retrieve a list of browsers/agents installed in client system

I would like to write a web application (in django) which scans the client/remote computers (assumption is windows) and retrieve the list of software's(mainly browsers) installed. Looking for suggestions to implement it.
Is this possible without asking the user to download any scritps/exe's?
If so, is it possible via java script?
I am planning to use python/django to write the entire app. Any input would be much appreciated.
EDIT : Comments on feasibility in java also much appreciated

Short answer: No, it is not possible
Long answer: This is something that any sane (operating) system designer / administrator would try to prevent - scanning of local system by a web page. However, you could use a plug-in component, such as a java applet, to do so - but in practice you probably would need to handle each client platform (OS) separately, since each of them has a different way of storing the information of installed software

You want to access the data from the client side so from the conceptual/logically its not good to access the client system. You have to use some medium which run on client side on behalf of server.
JavaScript and JavaApplet is good in this. You can get the data by JavaScript or Applet and in backend you can send data to the server.

You cannot do this unless you have some signed control installed on the client computer; or have them download a program which runs (separate from a browser) and sends the information to your server, where your django app can access it.
This is not possible using javascript (as it runs in a sandbox).

"Scanning" a client from a server may be possible if you break their security or get them to break it for you through some extension (see windows udate, for example). Either way, it's evil.

Cross-compiling Java to JavaScript

I have developed a Java applet which opens a URL connection to some different server. The applet then retrieves contents of the HTML page and do some processing and then shows to user. Can I cross-compile it to JavaScript using GWT?

Cross compile: No.
Port: Probably. Depends on your constraints.
You won't be able to do a straight recompile and have it "just work" (GWT only supports a subset of the JRE and any UI stuff definitely isn't a part of it) but you might be able to port some of your logic over. If you're using XPath to pull content out of the page, that code most likely will need to be redone as well. There's a GWT wrapper for Sarissa that works pretty well.
Also, since the requested page is going to be on a different server, you'll need to set up some method of doing a cross site request. So either browser hacks or a proxy on the hosting server.

How can I open a webpage within a Java app and run my own javascript code

I would like to open a webpage and run a javascript code from within a java app.
For example I would like to open the page www.mytestpage.com and run the following javascript code:
document.getElementById("txtEmail").value="test#hotmail.com";
submit();
void(0);
This works in a browser...how can I do it programatically within a java app?
Thanks!

You can use Rhino to execute JavaScript but you won't have a DOM available - i.e. document.getElementById() would work.
You can use HTML Unit (headless) or WebDriver/Selenium (Driving a browser) to execute JavaScript in an environment that has a DOM available.

I'm not sure what you are looking for but I assume that you want to write automated POST request. This can be done in with Http Client library. Only you have to set appropriate request (POST or GET) parameters.
Look at examples - with this library you can do basic authentication or post files too.

Your question is a bit ambiguous, as we don't know the position of the Java program.
If that's a Java applet inside your page, you should look at Java<->JavaScript interaction, it works well.
If you need a separate Java program to control a browser, like sending a bookmarklet in the address bar (as one of your tags suggests), it is a bit harder (depends on target browser), perhaps look at the Robot class.

There's Rhino JS engine written in Java that you can run on app server such as Tomcat and feed JS to, however - it's not clear what are you trying to do with this?
There's also Envjs simulated browser environment which is based on Rhino but complete enough to run jQuery and/or Prototype

DWR (and other frameworks) now support "reverse ajax." The general idea is that you use one of three methods to communicate back to the client:
Comet (long-lived https session)
Polling
opportunistic / piggy-back (i.e. next time a request comes from the client, append your js call)
Regardless of method (which is typically a configuration-time decision and not a coding issue), you will have full access to any/all js calls you want to make.
Check out the reference page from DWR to get a pretty good explanation.