I have a Java applet that runs on visiting a site, whereupon it receives, parses and displays streamed data. Unfortunately said applet isn't particularly well designed and I would like to extract the streamed data to make it more useful for myself.
In order to do this, I need to get insight into the connections made, data being exchanged, the state of variables in use and so on, so that I can replicate the log in and streaming.
So far I've been using Wireshark for network traffic, Java Debug Console and eyeballing the applet source installed on my system but still without success- primarily because there is a series of connections being made to the remote server and I am unsure the exact nature of the requests (and responses) and the variables contained.
Does a suitable app exist that can provide such insight?
I should add that I have no experience developing with Java, my knowledge is in Python, mainly.
Related
I am researching how to build an Intrusion Prevention System (IPS) application using Java, however I don't know how to redirect Internet packets from a PC to my application and read the packets. The data format I'm looking to get from the packets is similar to the data that Wireshark produces. How would I go about getting low-level packet data like this in a Java application?
Unfortunately, Java does not have access to "raw sockets"--the kind of access you need to get the details you'd expect from a Wireshark dump or the similar.
If you need to use Java for the rest of your application, consider using JNI (link is to a good JNI tutorial). It allows you to use C/C++/Fortran code within a Java application, so you can make system calls and get the kind of access you need.
I was going to use an objectOutputStream but heard this is unreliable because different java versions might deserialize objects differently. Something about 'horrible cross-architecture practice..'
So how else can I send objects and arrays between these devices, where the receiving end can piece back together the proper object or array data?
Edit: Just read what you are doing. You might not need a web server. A lot of people recommend one because of the massive support web servers have. You certainly use TCP or UDP to talk between a server and a client. You'll need to have some protocol if you want data interchange, and most people here would be familiar with XML or JSON
If you need inspiration, try looking at a few protocols like FTP, or even Bittorrent
Web Server case:
I wrote a Java web server for a college homework assignment. A web server actually be quite simple if you have a good grasp on TCP/IP. The code scattered everywhere online to do it though gets a little hard to decipher what exactly is going on, but once you do, it's not bad
You definitely should check out the RFC for HTTP, even though those tend to be worded in legalese. Beyond that, on the server, you basically read strings in line-by-line and you should be able to figure out what to do on the server (example GET /somefile.html HTTP/1.0). Just do System.out.println on those lines and go from there. The same goes for client code. You also can use telnet to see what a web server does
To test, I would actually recommend trying just a regular web browser like Firefox, Chrome, IE, Safari, and even curl scripts. This is an easy test to see if your server is running correctly
As far as data interchange goes, XML or JSON would be recommended, mostly that if you learn how to handle it, you get 100 experience points for your resume. However, to get things started, you can start out just by sending and receiving text like "Wazzzaaap". Web browsers can also grab XML and JSON data.
By 'java server', what kind of protocols are you using?
One option is RPC, which is defined in java.rmi
If you are using http, the simplest choice is to implement a small servlet in tomcat/jetty and use restful services
The data format can be xml, json, bin, etc
I am trying to set up a webserver on an old machine of mine. I have installed ubuntu server edition and aim to use it for the following:
I want to run a java program on the server. I want to be able to retrieve data from the program from another computer/phone using an internet connection. I also want to be able to give the program data, and get a response saying whether or not the data has been received correctly.
So for example:
A .jar program runs on my server and holds a variable x
I want to be able to query the value of x from another device (over the internet).
I want to be able to set the value of x remotely from another device, and get a response saying it was successful in altering the value.
What are my options here? I would like to try and keep things simple. It is perhaps worth mentioning that I will be the only one using the system. The server will be used exclusively for dealing with the two requests outline above.
Is it simply the case of creating a java program that listens out for incoming requests and running that on the server?
As you mentioned, you can start with custom ServerSocket wrapper which will decode incoming requests and do as it's bid. Currently, whole frameworks are done to encapsulate common code of this task -- see my 3rd point.
Old-school java solution: use RMI. See RMI tutorial.
New-school java solution: devise some simple text-based protocol with 2 commands:
Read()
Set(newVal)
Then implement that protocol over some new trendy Java framework, like Apache MINA, which is created specifically to facilitate quick development of network apps in Java.
I, personally, started with RMI for such kind of tasks. Since RMI is considered Core Java technology, it's wise to learn it.
Now there is a new requirement. I have got some adhoc work at hand. The requirement is to connect a desktop based Java application to read data from Mainframe generated by some CICS Transaction. [Basically I have to read all the records being appended into a file (same way as we do tail-f filename in linux). This is just for FYI my requirement is something different.]
I inquired, and came to know that my employer cannot provide MQ or CICS Transaction Gateway access to me. He suggested some method of screen scraping. I have already done that using VB.Net application and Quick3270 as well as IBM Communicator Emulators. Both these emulators provide functions which can be used to read whatever is there on the screen.
You can refer to EHILLAPI programming details (Language for Emulator programming) - http://publib.boulder.ibm.com/infocenter/pcomhelp/v5r9/index.jsp?topic=/com.ibm.pcomm.doc/books/html/emulator_programming07.htm if you are interested in learning.
But this method is restricting me to the maximum number of bytes that can come on the screen. With this method there is significant network delay as I have to refresh (basically move from one page to another on CICS) everytime to get data which is spanning across multiple pages.
Can you suggest me some method so that the my employer does not need to ask the client to open any port on his Mainframe or install any software (as this is not possible for my employer).
Can I use the 3270 terminal emulation and retrieve all (or at least more data). This way the requirement of my employer is fulfilled and he does not need to ask anything to his client. (In any case from the emulator we are firing CICS Transactions). We want everything to be done at my employer's end itself without disturbing the client's Mainframe even a single bit.
Please do not suggest MQ as the client does not have it.
If you are still suggesting CICS Transaction Gateway, then please let me know how would I connect to the remote machine (I need technical details).
- What information do I need to ask from the client.
- What software do I need to install on my machine.
- Technical details of using that software.
Regards,
Nitin
I have two suggestions for you to look at. I have done both successfully. Your client setup can decide if either is palatable (the question doesn't mention not doing these things).
You can call your CICS code on the mainframe via a DB2 stored procedure. There is a standard one supplied by IBM called EXECCICS that we used for a project. You supply the standard CICS parameters and comm area. The stored procedure executes the program in the mainframe and returns you the comm area. You use JDBC. This solution is simple and easy the execute.
We have also enabled HTTP access to the CICS program on the mainframe. To my understanding (remember I just called it -- not enabled it) it is a pretty standard configuration. The client code just performs an HTTP POST to a specific end point. The resulting document is the comm area plus other goodies.
These solutions were developed independently for the same project and are both in production. The only reason the HTTP method was added to the mix was because of a data size limit in the stored procedure that HTTP removed.
I am implementing a website using PHP for the front end and a Java service as the back end. The two parts are as follows:
PHP front end listens to http requests and interacts with the database.
The Java back end run continuously and responds to calls from the front end.
More specifically, the back end is a daemon that connects and maintain the link to several IM services (AOL, MSN, Yahoo, Jabber...).
Both of the layers will be deployed on the same system (a CentOS box, I suppose) and introducing a middle layer (for instance: using XML-RPC) will reduce the performance (the resource is also rather limited).
Question: Is there a way to link the two layers directly? (no more web services in between)
Since this is communication between two separate running processes, a "direct" call (as in JNI) is not possible. The easiest ways to do such interprocess communcation are probably named pipes and network sockets. In both cases, you'll have to define a communication protocol and implement it on both sides. Using a standard protocol such as XML-RPC makes this easier, but is not strictly necessary.
There are generally four patterns for application integration:
via Filesystem, ie. one producers writes data to a directory monitored by the consumer
via Database, ie. two applications share a schema or table and use it to swap data
via RMI/RPC/web service/any blocking, sync call from one app to another. For PHP to Java you can pick from the various integration libraries listed above, or use some web services standards like SOAP.
via messaging/any non-blocking, async operation where one app sends a message to another app.
Each of these patterns has pros and cons, but a good rule of thumb is to pick the one with the loosest coupling that you can get away with. For example, if you selected #4 your Java app could crash without also taking down your PHP app.
I'd suggest before looking at specific libraries or technologies listed in the answers here that you pick the right pattern for you, then investigate your specific options.
I have tried PHP-Java bridge(php-java-bridge.sourceforge.net/pjb/) and it works quite well. Basically, we need to run a jar file (JavaBridge.jar) which listens on port(there are several options available like Local socket, 8080 port and so on). Your java class files must be availabe to the JavaBridge in the classpath. You need to include a file Java.inc in your php and you can access the Java classes.
Sure, there are lots of ways, but you said about the limited resource...
IMHO define your own lightweight RPC-like protocol and use sockets on TCP/IP to communicate. Actually in this case there's no need to use full advantages of RPC etc... You need only to define API for this particular case and implement it on both sides. In this case you can serialize your packets to quite small. You can even assign a kind of GUIDs to your remote methods and use them to save the traffic and speed-up your intercommunication.
The advantage of sockets usage is that your solution will be pretty scalable.
You could try the PHP/Java integration.
Also, if the communication is one-way (something like "sendmail for IM"), you could write out the PHP requests to a file and monitor that in your Java app.
I was also faced with this problem recently. The Resin solution above is actually a complete re-write of PHP in Java along the lines of JRuby, Jython and Rhino. It is called Quercus. But I'm guessing for you as it was for me, tossing out your Apache/PHP setup isn't really an option.
And there are more problems with Quercus besides: the free version is GPL, which is tricky if you're developing commercial software (though not as tricky as Resin would like you to believe (but IANAL)) and on top of that the free version doesn't support compiling to byte code, so its basically an interpreter written in Java.
What I decided on in the end was to just exchange simple messages over HTTP. I used PHP's json_encode()/json_decode() and Java's json-lib to encode the messages in JSON (simple, text-based, good match for data model).
Another interesting and light-weight option would be to have Java generate PHP code and then use PHP include() directive to fetch that over HTTP and execute it. I haven't tried this though.
If its the actual HTTP calls you're concerned about (for performance), neither of these solutions will help there. All I can say is that I haven't had problems with the PHP and Java on the same LAN. My feeling is that it won't be a problem for the vast majority of applications as long as you keep your RPC calls fairly course-grained (which you really should do anyway).
Sorry, this is a bit of a quick answer but: i heard the Resin app server has support for integrating java and PHP.
They claim they can smash php and java together: http://www.caucho.com/resin-3.0/quercus/
I've used resin for serving J2ee applications, but not for its PHP support.
I'd be interested to hear of such adventures.
Why not use web service?
Make a Java layer and put a ws access(Axis, SpringWS, etc...) and the Php access the Java layer using one ws client.
I think it's simple and useful.
I've come across this page which introduces a means to link the two layers. However, it still requires a middle layer (TCP/IP). Moreover, other services may exploit the Java service as well because it accepts all incoming connections.
http://www.devx.com/Java/Article/20509
[Researching...]