JMeter proxy and java serialization in HTTP/POST? - java

We have an applet-servlet communication that we'd like to record with JMeter's HTTP proxy.
It works with GET messages until the applet sends an HTTP POST message which includes some serialized Java objects (built-in types), then we get this error in the Applet:
alt text http://img339.imageshack.us/img339/9238/appletservletjmeterhttp.png
OK, so there's some JVM version conflict somewhere in the queue. But where?
The communication runs OK without JMeter, that is: Applet -> Tomcat -> Servlet. All on my local machine.
But it doesn't work through JMeter: Applet -> JMeter proxy -> Tomcat -> Servlet. Also all on my machine.
It is as if JMeter was modifying the POST message content...
I tested it with the Apache proxy as well, working fine.
Even funnier thing is that I have only one version of Java installed, one JDK and one JRE. Both 1.6.0_07...
Thought I'd ask before starting digging deeper in the rabbit hole ;-)
Here is the hex dump of the POST data sent directly to Tomcat:
00000348 ac ed 00 05 73 72 00 11 6a 61 76 61 2e 6c 61 6e ....sr.. java.lan
00000358 67 2e 49 6e 74 65 67 65 72 12 e2 a0 a4 f7 81 87 g.Intege r.......
00000368 38 02 00 01 49 00 05 76 61 6c 75 65 78 72 00 10 8...I..v aluexr..
00000378 6a 61 76 61 2e 6c 61 6e 67 2e 4e 75 6d 62 65 72 java.lan g.Number
00000388 86 ac 95 1d 0b 94 e0 8b 02 00 00 78 70 00 00 01 ........ ...xp...
00000398 7b {
And here is the data when sent through JMeter:
00000128 ac ed 00 05 73 72 00 11 6a 61 76 61 2e 6c 61 6e ....sr.. java.lan
00000138 67 2e 49 6e 74 65 67 65 72 12 e2 a0 a4 f7 3f 3f g.Intege r.....??
00000148 38 02 00 01 49 00 05 76 61 6c 75 65 78 72 00 10 8...I..v aluexr..
00000158 6a 61 76 61 2e 6c 61 6e 67 2e 4e 75 6d 62 65 72 java.lan g.Number
00000168 3f ac 3f 1d 0b 3f e0 3f 02 00 00 78 70 00 00 01 ?.?..?.? ...xp...
00000178 7b {
A lot of "3f"s in the second dump...
So this is definitely some kind of an encoding problem.
The content type is set correctly in the header:
POST /ABCOrder/ABCServlet?cmd=getNetworkConnection HTTP/1.1
Connection: keep-alive
Content-Type: application/octet-stream
Host: 109.107.148.164:8443
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
User-Agent: Mozilla/4.0 (Windows Vista 6.0) Java/1.6.0_14
Content-Length: 81

Here is the solution:
JMeter has a config file, bin/jmeter.properties.
Here you can find an option where you can set the binary content types:
# Binary content-type handling
# These content-types will be handled by saving the request in a file:
proxy.binary.types=application/x-amf,application/x-java-serialized-object
Now I don't know why application/octet-stream isn't included by default, but you can simply add it to the list, and you are done.
proxy.binary.types=application/x-amf,application/x-java-serialized-object,application/octet-stream
This is how I found it out:
https://issues.apache.org/bugzilla/show_bug.cgi?id=44808
Did a search on JMeter closed bugs... :-)

Someone else is reporting a very similar: http://markmail.org/message/pl5erin2isehm5q6. I can't find any issue related to this problem in their bug tracker though. It looks like you won the privilege to dig deeper in the rabbit hole :)

The accepted answer only allows recording static requests.
This will not be realistic as it will not allow any variabilisation of requests (for example changing the searched word, ...) so you will always be stress testing the same bunch of data.
To make it a real test, you need to use a third party plugin.
A commercial JMeter plugin allows this, see:
http://ubikloadpack.com/
To make your tests realistic, you will need to variabilize content in the serialized objects.
This Java Serialization plugin will allow the following:
Easy recording of traffic with JMeter Proxy Server, a Test Plan using custom Sampler will be created
Easy variabilization of requests (which will appear as XML) through as easy syntax as for example ${searchedWord} where searchedWord can come from a CSV or any user defined variable.
Easy extraction of data from responses using JMeter standard Post Processors
Easy debugging of Request/Responses through standard JMeter View Results Tree element
Disclaimer :I work for this company.

Related

Port 0 in DIIOP_IOR.TXT, how do I change it?

I am trying to login remotely into Domino with standalone Java program.
I have ncso.jar (and TrustedCerts.class) in classpath.
The DIIOP_IOR.TXT file is generated by the diiop task. If I copy the file contents directly into my program and try creating the session like this:
String ior = "IOR:....." // 404 bytes
Session session = NotesFactory.createSessionWithIOR(ior, "username", "password");
the result is:
org.omg.CORBA.COMM_FAILURE: java.net.ConnectException: connect: Address is invalid on local machine, or port is not valid on remote machine Host: poseidon.heeros.com Port: 0 vmcid: 0x0 minor code: 1 completed: No
The server name is valid but port 0 seems odd. I tried an online decoder at http://www2.parc.com/istl/projects/ILU/parseIOR/ and here is the result:
object key is <#048525651a-ec68-106c-eee0-007e2d2233b5#00LotusNOI#01#00#01>;
no trustworthy most-specific-type info; unrecognized ORB type;
reachable with IIOP 1.1 at host "poseidon.heeros.com", port 0
...which seems to confirm that the port is incorrect. I have specified the server URL in Internet Sites with an IIOP Site document but there is no field for port there.
Questions:
Where do I set the port that appears in diiop_ior.txt?
Which port should I specify? (I'm guessing 1352)
EDIT
Here is the result of tell diiop show config on server:
Dump of Domino IIOP (DIIOP) Configuration Settings
Full Server Name: CN=Afrodite/O=Heeros
Common Server Name: Afrodite/Heeros
Refresh Interval: 3 minutes
Host Full Name: poseidon.heeros.com
Host Short Name: poseidon
Host Address: 10.163.0.146
Public Host Name/Address: poseidon.heeros.com
TCP Port: 0 Disabled
SSL Port: 63149 Enabled
Initial Net Timeout: 120 seconds
Session Timeout: 60 minutes
Client Session Timeout: 62 minutes
Allow Ambiguous Names: True
Web Name Authentic: False
User Lookup View: ($Users)
Allow Database Browsing: False
Internet Sites: Enabled
Internet Site Name: Heeros
Site Config Loaded from: Domino IIOP and Web Internet Site documents
Site is Default: False
Site Public Host Name/Address: poseidon.heeros.com
Site IOR File: D:\Lotus\Domino\data\domino\html\diiop_ior.txt
Site SSL Key File: D:\Lotus\Domino\data\heeros.kyr
Site Java Key File: D:\Lotus\Domino\data\domino\java\TrustedCerts.class
Site TCP Name/Password Allowed: False
Site TCP Anonymous Allowed: False
Site SSL Name/Password Allowed: True
Site SSL Anonymous Allowed: True
Site Multi-Server Session Authentication: Enabled
Site Multi-Server Session Configuration: LtpaToken
Single Server Cookies: Disabled
It seems that the correct port number is 63148. It must be specified in Server Document at Ports --> Internet Ports --> DIIOP as "TCP/IP port number".
Additionally, in the IIOP Site document, the TCP Authentication must be allowed.
In my experience DIIOP doesn't use SSL/TLS at all. Only the DIIOP_IOR.TXT is downloaded via SSL/TLS. Capture your network traffic with Wireshark or something similar and monitor: port 63148 or port 63149. #lauri-laanti : Could you please test in your environment if the connection is encrypted with wireshark?
Wireshark Ourput: GIOP createSession with Username and Password (Blanked with X)
0000 00 50 56 69 f5 2b 00 50 56 c0 00 02 08 00 45 00 .PVi.+.PV.....E.
0010 00 c0 0d 06 40 00 80 06 bb ca c0 a8 58 01 c0 a8 ....#.......X...
0020 58 15 d2 e0 f6 ac ef b6 47 e8 13 10 53 10 50 18 X.......G...S.P.
0030 01 00 29 bb 00 00 47 49 4f 50 01 00 00 00 00 00 ..)...GIOP......
0040 00 8c 00 00 00 00 00 00 00 05 01 00 00 00 00 00 ................
0050 00 31 04 38 35 32 35 36 35 31 61 2d 65 63 36 38 .1.8525651a-ec68
0060 2d 31 30 36 63 2d 65 65 65 30 2d 30 30 37 65 32 -106c-eee0-007e2
0070 64 32 32 33 33 62 35 00 4c 6f 74 75 73 4e 4f 49 d2233b5.LotusNOI
0080 01 00 01 00 00 00 00 00 00 0e 63 72 65 61 74 65 ..........create
0090 53 65 73 73 69 6f 6e 00 00 00 00 00 00 00 00 00 Session.........
00a0 00 01 00 00 00 00 00 00 00 01 00 00 00 0f 00 00 ................
00b0 00 06 00 61 00 64 00 6d 00 69 00 6e 00 00 00 00 ...a.d.m.i.n....
00c0 00 06 00 XX XX XX XX XX XX XX XX XX XX 00 ...XXXXXXXXXX.
Java Code used:
_diiop_args = new String[]{"-ORBEnableSSLSecurity", "-HTTPEnableSSLSecurity"};
String ior = NotesFactory.getIOR(_diiop_host + ":" + _diiop_port,
_diiop_args, _user_name, _user_pass);
_session = NotesFactory.createSessionWithIOR(ior, _user_name, _user_pass);

How to get SSL Domino Session via DIIOP

I try to connect to a Domino Server with a remote Java application started from Eclipse. The Domino Server allows SSL connections only.
I try to get the session with the following code.
String[] arg = new String[1];
arg[0] = "-ORBEnableSSLSecurity";
String IOR = NotesFactory.getIOR(DOMINO_SERVER);
session = NotesFactory.createSessionWithIOR(IOR);
I get the following error message:
Could not get IOR from Domino Server: http:///diiop_ior.txt
I also checked if the URL works in a browser. If I put the URL in a browser I get the correct response from the server.
The TrustedCert.class from the Domino server is included in my Eclipse project.
Here some configuration details from the "diiopcfg.txt":
TCP Port: 0 Disabled
SSL Port: 63149 Enabled
Site TCP Name/Password Allowed: True
Site TCP Anonymous Allowed: False
Site SSL Name/Password Allowed: True
Site SSL Anonymous Allowed: False
Site Multi-Server Session Authentication: Disabled
[Update]
Enabled TCP Port 63148, now I get a session but cannot open a database. Error message NotesException: Database open failed () Only when I access the port directly I get a session object.
[Update 2]
Get the session now. Can't open the database.
Error message: NotesException: Database ... has not been opened yet.
If I use the "open" method of the Database object => Error message: Database open failed()
Database db = session.getDatabase(DOMINO_SERVER, DOMINO_DATABASE);
db.open();
ACL is correct, Maximum internet name and password = Reader
Any idea why the database could not be opened. Tried another database with the same result.
Try the following to connect to SSL.
String args[] = new String[1];
args[0] = "-ORBEnableSSLSecurity";
Session s = NotesFactory.createSession(host, args, user, pwd);
Another method to connect:
String args[] = new String[1];
args[0] = "-HTTPEnableSSLSecurity";
String ior = NotesFactory.getIOR(host,args);
s = NotesFactory.createSessionWithIOR(ior, user, pwd);
The variable host should just be the host name and nothing else. Your diiop_ior.txt needs to be visible on SSL though (so check that first).
Alternatively you can try accessing the port 63148 directly. For example.
s = NotesFactory.createSession( "server:63148", user, pwd);
But this can move depending on server configuration.
Lastly you can pull the DIIOP_IOR.txt and use it directly. Same issue as previous alternative though.
Sorry, this answer is almost a duplicate to this answer but is so important, that I think the text needs to be quoted in this post.
In my experience DIIOP doesn't use SSL/TLS at all. Only the DIIOP_IOR.TXT is downloaded via SSL/TLS. Capture your network traffic with Wireshark or something similar and monitor: port 63148 or port 63149. #michael-schlömp : Could you please test in your environment if the connection is encrypted with wireshark?
Wireshark Ourput: GIOP createSession with Username and Password (Blanked with X)
0000 00 50 56 69 f5 2b 00 50 56 c0 00 02 08 00 45 00 .PVi.+.PV.....E.
0010 00 c0 0d 06 40 00 80 06 bb ca c0 a8 58 01 c0 a8 ....#.......X...
0020 58 15 d2 e0 f6 ac ef b6 47 e8 13 10 53 10 50 18 X.......G...S.P.
0030 01 00 29 bb 00 00 47 49 4f 50 01 00 00 00 00 00 ..)...GIOP......
0040 00 8c 00 00 00 00 00 00 00 05 01 00 00 00 00 00 ................
0050 00 31 04 38 35 32 35 36 35 31 61 2d 65 63 36 38 .1.8525651a-ec68
0060 2d 31 30 36 63 2d 65 65 65 30 2d 30 30 37 65 32 -106c-eee0-007e2
0070 64 32 32 33 33 62 35 00 4c 6f 74 75 73 4e 4f 49 d2233b5.LotusNOI
0080 01 00 01 00 00 00 00 00 00 0e 63 72 65 61 74 65 ..........create
0090 53 65 73 73 69 6f 6e 00 00 00 00 00 00 00 00 00 Session.........
00a0 00 01 00 00 00 00 00 00 00 01 00 00 00 0f 00 00 ................
00b0 00 06 00 61 00 64 00 6d 00 69 00 6e 00 00 00 00 ...a.d.m.i.n....
00c0 00 06 00 XX XX XX XX XX XX XX XX XX XX 00 ...XXXXXXXXXX.
Java Code used:
_diiop_args = new String[]{"-ORBEnableSSLSecurity", "-HTTPEnableSSLSecurity"};
String ior = NotesFactory.getIOR(_diiop_host + ":" + _diiop_port,
_diiop_args, _user_name, _user_pass);
_session = NotesFactory.createSessionWithIOR(ior, _user_name, _user_pass);
If you download and analyse the DIIOP_IOR.TXT with the ILU IOR Parser you will see there is no SSL/TLS information nor port in the IOR File.
SSL/TLS only version:
object key is <#048525651a-ec68-106c-eee0-007e2d2233b5#00LotusNOI#01#00#01>;
no trustworthy most-specific-type info; unrecognized ORB type;
reachable with IIOP 1.1 at host "testdom01.jjtest.site", port 0
SSL/TLS and non SSL/TLS version:
object key is <#048525651a-ec68-106c-eee0-007e2d2233b5#00LotusNOI#01#00#01>;
no trustworthy most-specific-type info; unrecognized ORB type;
reachable with IIOP 1.1 at host "testdom01.jjtest.site", port 63148

Java SSL Streaming - splitted applicationdata

I try to send a byte[] () over a established SSL Connection (handshake etc is done).
The result: The byte[] is spitted into two packets (see debug below):
First packet: just the first byte of the application data (**01**) .
Second packet: the rest (fe db 01 00 ...) 650 Bytes
Is there a way to commit all application data bytes in one packet?
Stream to send 651 Bytes:
**01** fe db 01 00 00 02 83 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 75 73 2d 61 73 63 69 69 22 20 73 74 61 6e 64 61 6c 6f 6e 65 3d 22 6e 6f 22 3f 3e …
javax.net.debug output
Padded plaintext before ENCRYPTION: len = 32
0000: **01** 06 03 06 46 7F 7F AE D4 E8 30 5D B7 DB 3C 44 ....F.....0]..<D
0010: 02 08 C9 2A A1 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A ...*............
1, WRITE: TLSv1 Application Data, length = 32
[Raw write]: length = 37
0000: 17 03 01 00 20 B3 4E EE CE 5B 69 EC A5 4A 80 7F .... .N..[i..J..
0010: D6 03 35 AF 6A 7B 85 17 B7 46 A2 31 B2 EF 7E D0 ..5.j....F.1....
0020: EA 1B 67 7E ED ..g..
Padded plaintext before ENCRYPTION: len = 672
0000: FE DB 01 00 00 02 83 3C 3F 78 6D 6C 20 76 65 72 .......<?xml ver
0010: 73 69 6F 6E 3D 22 31 2E 30 22 20 65 6E 63 6F 64 sion="1.0" encod
0020: 69 6E 67 3D 22 75 73 2D 61 73 63 69 69 22 20 73 ing="us-ascii" s
0030: 74 61 6E 64 61 6C 6F 6E 65 3D 22 6E 6F 22 3F 3E tandalone="no"?>
[…]
Sun's impl comments:
By default, we counter chosen plaintext issues on CBC mode
ciphersuites in SSLv3/TLS1.0 by sending one byte of application
data in the first record of every payload, and the rest in
subsequent record(s). Note that the issues have been solved in
TLS 1.1 or later.
Experiment with SSLEngine.wrap( largePlainText ) shows that it produces 2 SSL records, the 1st record contains 1 byte of plain text, the 2nd record contains 15846 bytes of plain text.
The receiver API probably handle record-by-record, so it'll return 1 byte for the 1st read.
We can also observe this behavior in other SSL impls, e.g. HTTPS requests from web browsers.
OpenSSL inserts empty records against the attack. If the receiver is Java SSL socket, the input stream cannot return 0 bytes for read(), so the record is skipped. Other receivers may not be prepared for a 0-length record and may break.
The assumption you're making about reading the byte[] exactly as you write them on the other end is a classic TCP mistake. It's not actually specific to SSL/TLS, but could also happen with a TCP connection.
There is no guarantee in TCP (and in SSL/TLS) that the reader's buffer will be filled with the exact same packet length as the packets in the writer's buffer. All TCP guarantees is in-order delivery, so you'll eventually get all your data, but you have to treat it as a stream.
This is why protocols that use TCP rely on indicators and delimiters to tell the other end when to stop reading certain messages.
For example, HTTP 1.1 uses a blank line to indicate when the headers end, and it uses the Content-Length header to tell the recipient what entity length to expect (or chunked transfer encoding). SMTP also uses line returns and . at the end of a message.
If you're designing your own protocol, you need to define a way for the recipient to know when what you define as meaningful units of data are delimited. When you read the data, read such indicators, and fill in your read buffer until you get the amount of bytes you expect or until you find the delimiter that you've defined.
I had the same problem until I saw this page:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7157903
So, I run the JVM with -Djsse.enableCBCProtection=false parameter and now the data is not splitted.
Best regards

How to detect Java string inside a .so file?

I'm recompiling a Java program, and I have the following method:
public static native String getMyString(String s);
getMyString is implemented inside mylibrary.so, and looks like follows:
; Scrambler::getMyString(char const*)
EXPORT _ZN9Scrambler18getMyStringEPKc
_ZN9Scrambler18getMyStringEPKc
LDR R1, =(mystring - 0x14FC8)
PUSH {R4-R6,LR}
ADD R1, PC ; "mystring"
MOV R4, R0
BLX strcasecmp
CMP R0, #0
BNE.W loc_150FC
which than ends up in
LDR R5, =(unk_6AE24 - 0x14FD8)
ADD R5, PC
The R5 seems to be the return value here.
The unk_6AE24 looks like follows:
27
68 5D 6D 06 3F 10 6D 26 32 10 56 12 06 3F 32 5D
12 27 26 4B 0D 5D 27 4B 6E 0D 2B 06 32 5D 68 24
3F 32 06 24 70 56 00 63 69 4C 1D 06 5A 7F 0C 34
1E 67 3B 63 32 5F 16 1D 34 5F 1E 32 42 69 18 49
3F 51 49 0C 1F 0C 0E 77 2F 44 69 7F 5A 0C 0B 34
0C 34 1C 24 32 5F 7F 00 00 00 00 1D 30 3B 5F 30
IDA suggests, that the string in question is 27 68 5D 6D or "'h]m", but I'm having my doubts. When I was using c++ ages ago, the strings used to end with 0, which would make for return value of 27 68 5D 6D 06 3F 10 6D 26 32 10 56 12 06 3F 32 5D 12 27 26 4B 0D 5D 27 4B 6E 0D 2B 06 32 5D 68 24 3F 32 06 24 70 56 . Or we could be talking about java string, which may have its own format.
What kind of string will getMyString return?
What kind of string will getMyString return?
It will return a java.lang.String, you can get that from the declaration.
The actual character data will (probably) be one redirection away from that. (a String is probably a pointer to a char array and a size). Keep in mind that in Java, a char is 16 bits.
What are you trying to do, anyway? Much easier to just set a breakpoint right after getMyString returns and look at the return value in a debugger.

Why might RvdProxy.getServices() incorrectly return an empty Array?

The following code snippet attempts to create a Tib DaemonManager connecting to a particular rvd, and then query for that rvd's services.
public static void main(String[] args) throws RuntimeException {
DaemonManager daemonManager = new DaemonManager("http://foo.com:7580");
if(daemonManager.getDaemonType() == DaemonManager.RVD) {
DaemonProxy daemonProxy = daemonManager.getDaemonProxy();
final RvdProxy rvdProxy = (RvdProxy) daemonProxy;
Service[] services = rvdProxy.getServices();
System.out.println(services.length); //prints 0
for (Service service : services) {
System.out.println(service.getNetwork());
}
}
}
This prints zero, even though the web interface for this rvd lists multiple available services. Why might this happen?
The daemon I am connecting to is running v 7.5.1 of the software, and the rvconfig.jar that I am using is from v 7.5.1 as well.
Is there a gotcha when using Tibco's DaemonManager that is causing me to come unstuck?
I used Wireshark to look at the traffic being sent between my RvdProxy and the RVD itself, and it looks like a lot of HTTP GET traffic. For example:
0000 00 1e 0b a4 d1 7c 00 12 d9 7c 8a bf 08 00 45 00 .....|.. .|....E.
0010 03 87 a4 58 40 00 3d 06 66 90 0a 09 14 15 0a 0a ...X#.=. f.......
0020 07 61 1d 9c bf 09 fe 1e d6 82 6e 77 b9 52 80 18 .a...... ..nw.R..
0030 00 1b 05 43 00 00 01 01 08 0a ac 1c 6f 67 07 b2 ...C.... ....og..
0040 86 1f 72 3d 23 30 30 38 30 38 30 3e 3c 66 6f 6e ..r=#008 080><fon
0050 74 20 66 61 63 65 3d 48 65 6c 76 65 74 69 63 61 t face=H elvetica
0060 2c 41 72 69 61 6c 20 73 69 7a 65 3d 32 20 63 6f ,Arial s ize=2 co
0070 6c 6f 72 3d 23 46 46 46 46 46 46 3e 3c 62 3e 43 lor=#FFF FFF><b>C
Sure enough, upon further inspection, it turns out that the requests being made by the proxy are to the web pages served by the Daemon itself - so the proxy API is just a screen-scraper.
But why am I getting no services if they appear on the web pages that are being screen-scraped?
Switch on debugging:
System.setProperty("com.tibco.tibrv.config.debug","classes-to-debug")
And you find that the pattern being used to extract the services from the web-page...
/services,GET~~~~~\
service_detail\\?(\\d+)>\\1</a>.*?size=2>(\\d+\\.\\d+\\.\\d+\\.\\d+)</td>.*?size=2>
(\\d+)</td>.*?size=2>(\\d+)</td>~~~~~\
...fails to match anything! In my case it is because my service networks are not of the form (\\d+\\.\\d+\\.\\d+\\.\\d+) but rather of the form ;(\\d+\\.\\d+\\.\\d+\\.\\d+) - note the leading semicolon. This subtle difference is the source of all of my problems!
This looks like a bug in the rvconfig jar - which needs to be raised with Tibco! :(

Categories

Resources