I am using derby in-memory db in java . And I want to run a sql. My sql string's length is 61671 byte. But I get Java class file format limit(s) exceeded: method:e1 code_length (72447 > 65535) error. I know the limitation about derby like code limit must be less than 65535 byte. It's ok (class file format limit(s) exceeded)
But now my string' length is just 61671. Why I am getting code_length (72447 > 65535) error ?
How can I fix it?
Thx
my string' length is just 61671. Why I am getting code_length (72447 > 65535) error ?
Because code length and string length are two different things. Your code is too long. Split it.
As others have said, your problem is that your code is too long, not any specific string. I am afraid splitting your code into smaller methods/classes is your only option. Not only that, but it's just good practice. It's good practice to break your code into as many different segments as reasonably possible. Often, if you end up with giant classes/methods, it's because your classes or methods have low cohesion.
High cohesion (and therefore ultimately small classes etc) is strongly encouraged (essential reading for object oriented programmers). It will make your life, and the life of any programmers working with you, much more comfortable.
If you're trying to store your classfiles in a Derby databse, perhaps you should use a BLOB datatype, which does not have such a small length limit.
Related
I have used the unpack data logic provided in below link for java
How to unpack COMP-3 digits using Java?
But for the null data in source it returns 404040404 like on Java unpack code. I understand this was space in ebcdic, but how to unpack by handling this space or to avoid it.
There are two problems that we have to deal with. First, is the data valid comp-3 data and second, is the data considered “valid” by older language implementations like COBOL since Comp-3 was mentioned.
If the offests are not misaligned it would appear that spaces are being interpreted by existing programs as 0 instead of spaces. This would be incorrect but could be an artifact of older programs that were engineered to tolerate this bad behaviour.
The approach I would take in a legacy shop (assuming no misalignment) is to consider “spaces” (which are sequences of 0x404040404040) as being zero. This would be a legacy check to compare the field with spaces and then assume that 0x00000000000f as the actual default. This is something an individual shop would have to determine and is not recognized as a general programming approach.
In terms of Java, one has to remember that bytes are “signed” so comparisons can be tricky based on how the code is written. The only “unsigned” data type I
recall in java is char which is really two bytes (unit 16) basically.
This is less of a programming problem than it is recognizing historical tolerance and remediation.
Question:
Instead of writing my own serialization algorithm; would it be possible to just use the built in Java serialization, like I have done below, while still having it work across multiple languages?
Explanation:
How I imagine it working, would be as follows: I start up a process, that will be be a language-specific program - written in that language. So I'd have a CppExecutor.exe file, for example. I would write data to a stream to this program. The program would then do what it needs to do, then return a result.
To do this, I would need to serialize the data in some way. The first thing that came to mind was the basic Java Serialization with the use of an ObjectInputStream and ObjectOutputStream. Most of what I have read has only stated that the Java serialization is Java-to-Java applications.
None of the data will ever need to be stored in a file. The method of transferring these packets would be through a java.lang.Process, which I have set up already.
The data will be composed of the following:
String - Mostly containing information that is displayed to the user.
Integer - most likely 32-bit. Won't need to deal with times.
Float- just to handle all floating-point values.
Character - to ensure proper types are used.
Array - Composed of any of the elements in this list.
The best way I have worked out how to do this is as follows: I would start with a 4-byte magic number - just to ensure we are working with the correct data. Following, I would have an integer specifying how many elements there are. After that, for each of the elements I would have: a single byte, signifying the data type (of the above), following by any crucial information, e.x: length for the String and Array. Then, the data that follows.
Side-notes:
I would also like to point out that a lot of these calculations will be taking place, where every millisecond could matter. Due to this, a text-based format (such as JSON) may produce far larger operation times. Considering that non of the packets would need to be interpreted by a human, using only bytes wouldn't be an issue.
I'd recommend Google protobuf: it is binary, stable, proven, and has bindings for all languages you've mentioned. Moreover, it also handles structured data nicely.
There is a binary json format called bson.
I would also like to point out that a lot of these calculations will be taking place, so a text-based format (such as JSON) may produce far larger operation times.
Do not optimize before you measured.
Premature optimization is the root of all evil.
Can you have a try and benchmark the throughput? See if it fits your needs?
Thrift,Protobuf,JSON,MessagePack
complexity of installation Thrift >> Protobuf > BSON > MessagePack > JSON
serialization data size JSON > MessagePack > Binary Thrift > Compact Thrift > Protobuf
time cost Compact Thrift > Binary Thrift > Protobuf > JSON > MessagePack
So, for example, Notification has the following flag:
public static final int FLAG_AUTO_CANCEL = 0x00000010;
This is hexadecimal for the number 16. There are other flags with values:
0x00000020
0x00000040
0x00000080
Each time, it goes up by a power of 2. Converting this to binary, we get:
00010000
00100000
01000000
10000000
Hence, we can use a bitwise operators to determine which of the flags are present, etc, since each flag contains only one 1 and they are all in different locations.
Question:
This all makes perfect sense, but why not just use booleans? Is this merely stylistic, or are there memory or efficiency benefits?
EDIT:
I understand that by combining them, we can store a lot of information in a single int. Is this used solely so we can pass a lot of boolean type values in a single int instead of having to pass a ton of parameters? I don't mean to trivialize that, it's very convenient, but are there any other benefits?
What you're talking about is called a Bit Field. One advantage is that all the information can be contained in a single variable (with no overhead like that of an ArrayList). This is useful for keeping function signatures tidy, and will have some minor benefits with efficiency because of fewer stack operations, but probably this will be offset by additional bitshift operations. Additionally, you can use (for example) one byte to store 8 fields rather than wasting 7 additional bytes. You can also, if you're clever with it, perform several flag checks in a single operation.
Having said that, personal preference may see the list of booleans as cleaner or preferable. Bitfields are most common in embedded systems where space is limited or something of that nature.
In reference to your edit: it's storing the values of the flags in ints, but those are just reference constants-- you aren't editing those, you're sticking those bits into (or out of) the flags field, which is a single int. I don't really know why they chose a bitfield for this application; perhaps someone that grew up programming space-limited microcontrollers coded that specific class. The general consensus seems to be that bitfields shouldn't be included in new code.
This is a common idiom in C, where resource constraints are a much larger concern, and you usually see it in Java where the Java API is directly mapping an underlying well-known C API. However, it's not a great idea in Java for a wide number of reasons.
As of Java 5, most of the uses for one-bit bit fields are taken care of very nicely by EnumSet, which is internally implemented using a bit field (so it's extremely fast) but is type-safe, easy to read, and Iterable.
How can null byte injection be done on a java webapp, Or rather - how does on protect against it?
Should I look at each byte of the request parameter and inspect its 'byte' value to be 0 ? I can't imagine a 0 byte sneaking in a request parameter... can it?
My main aim is to make sure the filename used for saving the file is safe enough. And for now, I am not looking answers that recommend (for example): replacing ALL non-word characters with Underscore.
Allowing the user to store files with arbitrary names is dangerous. What happens if the user provides "../../../WINDOWS/explorer.exe"? You should restrict filenames to only contain characters known to be harmless.
'\0' is not known to be harmless. As far as Java is concerned, '\0' is a character like any other. However, the operating system is likely to interpret '\0' as the end of a string. If a string is passed from Java to the operating system, that different interpretation could result in exploitable bugs. Consider:
if (filename.endsWith(".txt") {
store(filename, data);
}
where filename is "C:\Windows\explorer.exe\0.txt", which ends with ".txt" to Java, but with ".exe" to the operating system.
I'm not sure why you're concerned with null byte injection. Java isn't like C/C++, where strings are null-terminated character arrays.
You ought to bind and validate parameters and values coming in from the web tier. How do you define "safe enough"?
You have 2 choices:
1 Scan the string (convert it to a char array first) for null bytes.
2 upgrade to Java 8 or Java 7u40 and you are protected. (Yes, i tested it!, it works!)
in May 1013 Oracle fixed the problem: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8014846
Null byte injection in filenames was fixed in Java 7 update 40 (released around Sept. 2013). So, its been fixed for a while now, but it WAS a problem for over a decade and it was a NASTY vulnerability in Java. The fix is documented here: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8014846
-Dave Wichers
I read somewhere that the Java StringBuilder uses around 1 mb for 500 characters. Is this true and if so, isn't that a bit extreme? Is the StringBuilder doing some incredible things with this amount of memory? What is the reason and does this mean I should not make too much use of this class?
No, that's complete rubbish - unless you create a StringBuilder with a mammoth capacity, of course.
Java in general uses 2 bytes per char. There's a little bit of overhead in String and StringBuilder for the length and the array itself, but not a lot.
Now 1K for 500 characters is about right... I suspect that was the cause of confusion. (Either you misheard, or the person talking to you was repeating something they'd misheard.)
I have seen two cases where StringBuilder's tend to use large amounts of memory:
When the StringBuilder is created with an insane initial-capacity.
StringBuilder's who were "cached" to "save" object-allocation time.
So in the second case a StringBuilder might consume 1Mb of memory if some code, which used the SB earlier, stored a very big string in it. That's because it will only grow but not shrink its internal char-array.
Both cases can (and should) easy be avoided.
This information is erroneous, do you remember what the source of this information was? If so you should correct it. Java normally uses 2 bytes per character.
Because of the doubling reallocation 2K for 500 characters would also be right, but not more. Here is a similar question.
I think StringBuilder is the best choice to use. It is faster and safer too. It depends on the scenario. If you have String literal that doesn't change frequently then I would say String is a better choice because it is immutable else StringBuilder is right there. Now for the space that you are talking about I haven't heard that any where.