Develop Reference

Why Java GzipOutputStream always write two bytes 03 00 before the trailer if flush() is called first then closed?

I found that when using Java.util.zip.GzipOutputStream, when I called flush() first then close(). The deflator always produce two bytes ('00' and '03' in hex) before writing the trailer. What does these two bytes mean? Why flush() does not flush all bytes in the compressor? I have already set the stream syncFlush to true. (In my use case I need to force flush the stream)
This is the test code I write.
File file = new File("gtest.gz");
OutputStream out = new FileOutputStream(file);
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(out, true);
gzipOutputStream.write("Each gzip file generated by JAVA GzipOutputStream has byte '0003' before Trailer"
.getBytes());
gzipOutputStream.flush();
gzipOutputStream.close();
And I run hexdump -C gtest.gz get
00000000 1f 8b 08 00 00 00 00 00 00 00 72 4d 4c ce 50 48 |..........rML.PH|
00000010 af ca 2c 50 48 cb cc 49 55 48 4f cd 4b 2d 4a 2c |..,PH..IUHO.K-J,|
00000020 49 4d 51 48 aa 54 f0 72 0c 73 54 70 07 ca f9 97 |IMQH.T.r.sTp....|
00000030 96 14 94 96 04 97 14 a5 26 e6 2a 64 24 16 03 25 |........&.*d$..%|
00000040 4b 52 15 d4 0d 0c 0c 8c d5 15 92 52 d3 f2 8b 52 |KR.........R...R|
00000050 15 42 8a 12 81 06 14 01 00 00 00 ff ff 03 00 96 |.B..............|
00000060 ef 3b 5d 50 00 00 00 |.;]P...|
00000067
The last 8 bytes are trailer of Gzip and the two bytes before '03' '00' will always be produced whatever content I change.

Where does InstallCert.java get the server certificates

I'm attempting to follow the instructions on this page:
http://www.mkyong.com/webservices/jax-ws/suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/
to create a certificate for my localhost in which to do some development testing.
When running InstallCert for localhost:8443, the following two certificates are generated:
Server sent 2 certificate(s):
1 Subject CN=localhost4.localdomain4, O=example.com, C=US
Issuer CN=Certificate Shack, O=example.com, C=US
sha1 f4 2a a9 09 32 a6 ee 41 9d 9c 44 e6 4a bc 31 79 17 cb 88 fd
md5 e0 78 65 83 30 33 78 c5 80 17 e7 7a a2 91 85 52
2 Subject CN=Certificate Shack, O=example.com, C=US
Issuer CN=Certificate Shack, O=example.com, C=US
sha1 b8 87 d6 2d ac d8 36 06 7c 58 68 10 3e 21 39 6a a0 33 a1 25
md5 07 24 57 5f f8 35 1e 97 70 ff 54 aa 13 e6 6b 12
The trouble is that my system needs the CN to be localhost. I have no idea where the localhost4.localdomain4 comes from. How can I change this to be simply localhost?

The certificate comes from the server, during the handshake.
The CN is inside the certificate.
You can't change it without creating a new server certificate.

Java SSL Streaming - splitted applicationdata

I try to send a byte[] () over a established SSL Connection (handshake etc is done).
The result: The byte[] is spitted into two packets (see debug below):
First packet: just the first byte of the application data (**01**) .
Second packet: the rest (fe db 01 00 ...) 650 Bytes
Is there a way to commit all application data bytes in one packet?
Stream to send 651 Bytes:
**01** fe db 01 00 00 02 83 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 75 73 2d 61 73 63 69 69 22 20 73 74 61 6e 64 61 6c 6f 6e 65 3d 22 6e 6f 22 3f 3e …
javax.net.debug output
Padded plaintext before ENCRYPTION: len = 32
0000: **01** 06 03 06 46 7F 7F AE D4 E8 30 5D B7 DB 3C 44 ....F.....0]..<D
0010: 02 08 C9 2A A1 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A ...*............
1, WRITE: TLSv1 Application Data, length = 32
[Raw write]: length = 37
0000: 17 03 01 00 20 B3 4E EE CE 5B 69 EC A5 4A 80 7F .... .N..[i..J..
0010: D6 03 35 AF 6A 7B 85 17 B7 46 A2 31 B2 EF 7E D0 ..5.j....F.1....
0020: EA 1B 67 7E ED ..g..
Padded plaintext before ENCRYPTION: len = 672
0000: FE DB 01 00 00 02 83 3C 3F 78 6D 6C 20 76 65 72 .......<?xml ver
0010: 73 69 6F 6E 3D 22 31 2E 30 22 20 65 6E 63 6F 64 sion="1.0" encod
0020: 69 6E 67 3D 22 75 73 2D 61 73 63 69 69 22 20 73 ing="us-ascii" s
0030: 74 61 6E 64 61 6C 6F 6E 65 3D 22 6E 6F 22 3F 3E tandalone="no"?>
[…]

Sun's impl comments:
By default, we counter chosen plaintext issues on CBC mode
ciphersuites in SSLv3/TLS1.0 by sending one byte of application
data in the first record of every payload, and the rest in
subsequent record(s). Note that the issues have been solved in
TLS 1.1 or later.
Experiment with SSLEngine.wrap( largePlainText ) shows that it produces 2 SSL records, the 1st record contains 1 byte of plain text, the 2nd record contains 15846 bytes of plain text.
The receiver API probably handle record-by-record, so it'll return 1 byte for the 1st read.
We can also observe this behavior in other SSL impls, e.g. HTTPS requests from web browsers.
OpenSSL inserts empty records against the attack. If the receiver is Java SSL socket, the input stream cannot return 0 bytes for read(), so the record is skipped. Other receivers may not be prepared for a 0-length record and may break.

The assumption you're making about reading the byte[] exactly as you write them on the other end is a classic TCP mistake. It's not actually specific to SSL/TLS, but could also happen with a TCP connection.
There is no guarantee in TCP (and in SSL/TLS) that the reader's buffer will be filled with the exact same packet length as the packets in the writer's buffer. All TCP guarantees is in-order delivery, so you'll eventually get all your data, but you have to treat it as a stream.
This is why protocols that use TCP rely on indicators and delimiters to tell the other end when to stop reading certain messages.
For example, HTTP 1.1 uses a blank line to indicate when the headers end, and it uses the Content-Length header to tell the recipient what entity length to expect (or chunked transfer encoding). SMTP also uses line returns and . at the end of a message.
If you're designing your own protocol, you need to define a way for the recipient to know when what you define as meaningful units of data are delimited. When you read the data, read such indicators, and fill in your read buffer until you get the amount of bytes you expect or until you find the delimiter that you've defined.

I had the same problem until I saw this page:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7157903
So, I run the JVM with -Djsse.enableCBCProtection=false parameter and now the data is not splitted.
Best regards

How to detect Java string inside a .so file?

I'm recompiling a Java program, and I have the following method:
public static native String getMyString(String s);
getMyString is implemented inside mylibrary.so, and looks like follows:
; Scrambler::getMyString(char const*)
EXPORT _ZN9Scrambler18getMyStringEPKc
_ZN9Scrambler18getMyStringEPKc
LDR R1, =(mystring - 0x14FC8)
PUSH {R4-R6,LR}
ADD R1, PC ; "mystring"
MOV R4, R0
BLX strcasecmp
CMP R0, #0
BNE.W loc_150FC
which than ends up in
LDR R5, =(unk_6AE24 - 0x14FD8)
ADD R5, PC
The R5 seems to be the return value here.
The unk_6AE24 looks like follows:
27
68 5D 6D 06 3F 10 6D 26 32 10 56 12 06 3F 32 5D
12 27 26 4B 0D 5D 27 4B 6E 0D 2B 06 32 5D 68 24
3F 32 06 24 70 56 00 63 69 4C 1D 06 5A 7F 0C 34
1E 67 3B 63 32 5F 16 1D 34 5F 1E 32 42 69 18 49
3F 51 49 0C 1F 0C 0E 77 2F 44 69 7F 5A 0C 0B 34
0C 34 1C 24 32 5F 7F 00 00 00 00 1D 30 3B 5F 30
IDA suggests, that the string in question is 27 68 5D 6D or "'h]m", but I'm having my doubts. When I was using c++ ages ago, the strings used to end with 0, which would make for return value of 27 68 5D 6D 06 3F 10 6D 26 32 10 56 12 06 3F 32 5D 12 27 26 4B 0D 5D 27 4B 6E 0D 2B 06 32 5D 68 24 3F 32 06 24 70 56 . Or we could be talking about java string, which may have its own format.
What kind of string will getMyString return?

What kind of string will getMyString return?
It will return a java.lang.String, you can get that from the declaration.
The actual character data will (probably) be one redirection away from that. (a String is probably a pointer to a char array and a size). Keep in mind that in Java, a char is 16 bits.
What are you trying to do, anyway? Much easier to just set a breakpoint right after getMyString returns and look at the return value in a debugger.

Why might RvdProxy.getServices() incorrectly return an empty Array?

The following code snippet attempts to create a Tib DaemonManager connecting to a particular rvd, and then query for that rvd's services.
public static void main(String[] args) throws RuntimeException {
DaemonManager daemonManager = new DaemonManager("http://foo.com:7580");
if(daemonManager.getDaemonType() == DaemonManager.RVD) {
DaemonProxy daemonProxy = daemonManager.getDaemonProxy();
final RvdProxy rvdProxy = (RvdProxy) daemonProxy;
Service[] services = rvdProxy.getServices();
System.out.println(services.length); //prints 0
for (Service service : services) {
System.out.println(service.getNetwork());
}
}
}
This prints zero, even though the web interface for this rvd lists multiple available services. Why might this happen?
The daemon I am connecting to is running v 7.5.1 of the software, and the rvconfig.jar that I am using is from v 7.5.1 as well.
Is there a gotcha when using Tibco's DaemonManager that is causing me to come unstuck?

I used Wireshark to look at the traffic being sent between my RvdProxy and the RVD itself, and it looks like a lot of HTTP GET traffic. For example:
0000 00 1e 0b a4 d1 7c 00 12 d9 7c 8a bf 08 00 45 00 .....|.. .|....E.
0010 03 87 a4 58 40 00 3d 06 66 90 0a 09 14 15 0a 0a ...X#.=. f.......
0020 07 61 1d 9c bf 09 fe 1e d6 82 6e 77 b9 52 80 18 .a...... ..nw.R..
0030 00 1b 05 43 00 00 01 01 08 0a ac 1c 6f 67 07 b2 ...C.... ....og..
0040 86 1f 72 3d 23 30 30 38 30 38 30 3e 3c 66 6f 6e ..r=#008 080><fon
0050 74 20 66 61 63 65 3d 48 65 6c 76 65 74 69 63 61 t face=H elvetica
0060 2c 41 72 69 61 6c 20 73 69 7a 65 3d 32 20 63 6f ,Arial s ize=2 co
0070 6c 6f 72 3d 23 46 46 46 46 46 46 3e 3c 62 3e 43 lor=#FFF FFF><b>C
Sure enough, upon further inspection, it turns out that the requests being made by the proxy are to the web pages served by the Daemon itself - so the proxy API is just a screen-scraper.
But why am I getting no services if they appear on the web pages that are being screen-scraped?
Switch on debugging:
System.setProperty("com.tibco.tibrv.config.debug","classes-to-debug")
And you find that the pattern being used to extract the services from the web-page...
/services,GET~~~~~\
service_detail\\?(\\d+)>\\1</a>.*?size=2>(\\d+\\.\\d+\\.\\d+\\.\\d+)</td>.*?size=2>
(\\d+)</td>.*?size=2>(\\d+)</td>~~~~~\
...fails to match anything! In my case it is because my service networks are not of the form (\\d+\\.\\d+\\.\\d+\\.\\d+) but rather of the form ;(\\d+\\.\\d+\\.\\d+\\.\\d+) - note the leading semicolon. This subtle difference is the source of all of my problems!
This looks like a bug in the rvconfig jar - which needs to be raised with Tibco! :(