Develop Reference

How do I get a NetcdfFile out of a byte array?

I am relatively new to using the netcdf-java library, and I've immediately run into a problem when trying to load a file. The problem is that there doesn't seem to be a way to load a NetcdfFile from a byte array stored in memory, and that is the base form of my data. To elaborate a little, it is actually a .cdf file uploaded through a client, which the client then converts into a byte array for the server code to read. So the server, where my code is running, cannot see the uploaded file at all. I also cannot assume the server itself is writable, so essentially there is no "location" to pass into the typical NetcdfFile loading methods.
The FAQ on ucar.edu does mention the possibility of reading from a non-file source, here. It says I should write my own IOSP, which I am happy to do. However, there is very little guidance on how to do this.
I don't know how to implement isValidFile when the only thing passed into the function is a RandomAccessFile, which the FAQ says can be ignored.
I don't know how my IOSP will obtain the byte array in question for use in readData.
I don't know why the minimal example in the FAQ advises me to make a new NetcdfFile class, when it seems I could just use the default one but pass in my custom IOSP.
This question is a little vague, but I am truly lost without many clues on where to even begin. Any guidance would be appreciated.
EDIT: I'm using 5.4.2 of the netcdf-java library.

I found this answer in the support archives. The solution is to use InMemoryRandomAccessFile. The constructor takes a String location and a byte array containing the file's contents. From my testing, I think the location can be any arbitrary string. Here is the code that worked for me.
byte[] filebytes = retrieveFileBytes(clientFilepath);
InMemoryRandomAccessFile raf = new InMemoryRandomAccessFile(clientFilepath, filebytes);
NetcdfFile file = NetcdfFiles.open(raf, clientFilepath, null, null);
Variable peakRetentionTime = file.findVariable("peak_retention_time");
if (peakRetentionTime == null) {
displayWarning("peak_retention_time null!");
} else {
Array data = peakRetentionTime.read();
displayInfo(Ncdump.printArray(data));
}

Read Image, modify metadata, and re-write image in pure Java

I need to be able to update image metadata (namely, tags, creator, description, comments)
and do it within regular Exif and XMP. Most likely, i'll be reading the Exif, and writing XMP.
After much searching for a library that works ALSO for writing, I came across twelvemonkeys.
https://github.com/haraldk/TwelveMonkeys
This seemed to be promising. And indeed, with little effort I was able to read, already, the Exif containing description in one of my images. Not with the standard javax API, mind you, but with a twelvemonkeys API. That's fine with me. Whatever works!
At this point, I was happy to avoid the standard API as much as possible, as it seemed horribly convoluted and inefficient. I started about reading in my Exif, and coding the modification for my proof-of-concept. The idea being, the most efficient way to achieve what I want (quick and safe modification of metadata within JPEG files) was to perform the following steps:
Read all segments into a list
Find the segment that needs modification
Do that modification
Write all segments, sequentially, to a temp file
If all goes well, rename the original to later safely delete, rename the copy to the original name, and finally, delete the original file.
However, I was a bit dismayed when I discovered that there seems to be no implementation of
com.twelvemonkeys.imageio.metadata.Directory
which implements the methods
add(Entry)
and
remove(Object)
with anything other than a
throw new UnsupportedOperationException("Directory is read-only");
If this is not the way to efficiently (and safely) achieve what I want to do... Does anyone have a suggestion on how, in pure Java, I can do this?

Disclaimer: I designed and wrote the various metadata readers/writers mainly for internal use in my ImageIO library, and did not carefully consider use by third parties. For this reason, the API may not be "perfect" in this sense. But what you want to do should be perfectly doable. :-)
While the specific Directory implementations are indeed read-only, you can easily create your own subclass of AbstractDirectory that is mutable. Or just use any Collection<? extends Entry> you like and wrap it in a TIFFDirectory or IFD before writing. I prefer the latter, so I'll show that first.
Note that a typical JPEG Exif segment contains two IFDs, IFD0 for the main JPEG image, and IFD1 for the thumbnail. Thus you need to tread it as a CompoundDirectory:
CompoundDirectory exif = (CompoundDirectory) new TIFFReader().read(input);
List<Directory> ifds = new ArrayList<>;
for (int i = 0; i < exif.directoryCount(); i++) {
List<Entry> entries = new ArrayList<>();
for (Entry entry : exif.getDirectory(i)) {
entries.add(entry);
}
// TODO: Do stuff with entries, remove, add, change, etc...
ifds.add(new IFD(entries));
}
// Write Exif
new TIFFWriter().write(new TIFFDirectory(ifds), output);
You could also create your own mutable Directory:
public final class MutableDirectory extends AbstractDirectory {
public MutableDirectory (final Collection<? extends Entry> entries) {
super(entries);
}
public boolean isReadOnly() {
return false;
}
// NOTE: While the above is all you need to make it *mutable*,
// TIFF/Exif does not allow entries with duplicate IDs,
// you need to handle this somehow. The below code is untested...
#Override
public boolean add(Entry entry) {
Entry existing = getEntryById(entry.getIdentifier());
if (existing != null) {
remove(existing);
}
super.add(entry);
}
}
The reason for not implementing mutable directories is exactly that the semantics for how entries are handled may differ from format to format.

Reading and writing objects via GZIP streams?

I am new to Java. I want to learn to use GZIPstreams. I already have tried this:
ArrayList<SubImage>myObject = new ArrayList<SubImage>(); // SubImage is a Serializable class
ObjectOutputStream compressedOutput = new ObjectOutputStream(
new BufferedOutputStream(new GZIPOutputStream(new FileOutputStream(
new File("....")))));
compressedOutput.writeObject(myObject);
and
ObjectInputStream compressedInput = new ObjectInputStream(
new BufferedInputStream(new GZIPInputStream(new FileInputStream(
new File("....")))));
myObject=(ArrayList<SubImage>)compressedInput.readObject();
When the program writes myObject to a file without throwing any exception, but when it reaches the line
myObject=(ArrayList<SubImage>)compressedInput.readObject();
it throws this exception:
Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream
How can I solve this problem?

You have to flush and close your outputstream. Otherwhise, at least, the BufferedOutputStream will not write everything to the file (it does in big chucks to avoid penalizing performance).
If you call compressedOutput.flush() and compressedOutput.close() it will suffice.
You can try writing a simple string object and checking if the file is well written.
How? If you write a xxx.txt.gz file you can open it with your preferred zip app and look at the xxx.txt. If the app complains, then the content is not full written.
Extended answer to a comment: compressing even more the data
Changing serialization
You could change the standard serialization of SubImage object if it's an object of your own. Check java.io.Serializable javadoc to know how to do it. It's pretty straightforward.
Writing just what you need
Serialization has the drawback that needs to write "it's a SubImage" just before every instance you write. It's not necessary if you know what's going to be there beforehand. So you could try to serialize it more manually.
To write your list, instead of writing an object write directly the values that conform your list. You will need just a DataOutputStream (but ObjectOutputStream is a DOS so you can use it anyway).
dos.writeInt(yourList.size()); // tell how many items
for (SubImage si: yourList) {
// write every field, in order (this should be a method called writeSubImage :)
dos.writeInt(...);
dos.writeInt(...);
...
}
// to read the thing just:
int size = dis.readInt();
for (int i=0; i<size; i++) {
// read every field, in the same order (this should be a method called readSubImage :)
dis.readInt(...);
dis.readInt(...);
...
// create the subimage
// add it to the list you are recreating
}
This method is more manual but if:
you know what's going to be written
you will not need this kind of serialization for many types
it's pretty affordable and definitively more compressed than the Serializable counterpart.
Have in mind that there are alternative frameworks to serialize objects or create string messages (XStream for xml, Google Protocol Buffers for binary messages, and so on). That frameworks could work directly to binary or writing a string that could be then written.
If your app will need more on this, or just curious, maybe you should look at them.
Alternative serialization frameworks
Just looked in SO and found several questions (and answers) addressing this issue:
https://stackoverflow.com/search?q=alternative+serialization+frameworks+java
I've found that XStream is pretty easy and straightforward to use. And JSON is a format pretty readable and succint (and Javascript compatible which could be a plus :).
I should go for:
Object -> JSON -> OutputStreamWriter(UTF-8) -> GZippedOutputStream -> FileOutputStream

How to dynamically decompile a Class Object on memory?

I'm making a tool to dynamically display the sourcecode of running java class. I need a tool to help me on dynamically decompile from a Class Object to String of sourcecode. I know some decompile tools like Jad, DJ decompiler can decompile a .class file but I expect a tool can:
Class<?> c = ..; // get from runtime environment
String sourcecode = **DecompileTool**.decompileClassObject(c);
return sourcecode;
I need such a DecompileTool, anyone knows? Thanks

I'm not aware of any decompiler that can be used like that.
Indeed, in the general case it is not possible to implement a decompiler that works like that:
The Class<?> object that you get from the runtime doesn't provide any way to get to the bytecodes.
In order to get hold of the bytecodes, you would need to redo what the classloader does when it locates the ".class" file from the classpath.
I don't think there's a way to find out what classloaders are in use ... if you include the possibility of dynamically instantiated classloaders. (And such classloaders are normal practice in (for example) web containers.)
In the general case, a classloader will do that in ways that you cannot reproduce ... without reverse engineering and hard-coding the same logic into your decompiler adapter code.
Besides, doing this on the fly is probably pointless, because there is a significant chance that the decompiler will produce source code that isn't compilable.

I don't think that any of these decompilers support this type of ugly interface.
First of all, most decompilers will represent any code in a similar format to the actual compiler, so, an Abstract Syntax Tree. If you are lucky, and the decompiler does have an interface, it will probably be of this type. Handing back a raw String is unlikely to be satisfactory, because how would the person writing the decompiler have any idea as to how you wanted the code formatted (one of the biggest challenges in decompilation is presenting the result to the user!).
Instead, what you should do, is write a little wrapper, that does this properly: on the fly generation of the files that need to be decompiled, sending them through the decompiler, and extracting the result into a String (which you can get immediately if you do clever forking and piping, etc..., but in reality you probably just want to format the output file of the decompiler..)

Try Cavaj Java Decomplier, it may be useful for you.If you aren't satisfied this, try JadClipse with eclipse IDE.

You can do the followin thing steps
1) You can use decompilers available to decompile the code like
Process p = Runtime.getRuntime().exec("jad /location/CompilesClass.class");
BufferedReader in = new BufferedReader(
new InputStreamReader(p.getInputStream()));
String line = null;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
2) Now the Compiled class is converted into .jad extension (Source Code) in the path where the class file was there
3) Now you can read that .jad file by Scanner class of JDK6. Like
Scanner scanner = new Scanner(new File("locationOfJADFIle")).useDelimiter("\n");
while(scanner.hasNext()){
System.out.println(scanner.next());
}

What are the best/simplest classes used for WRITING files in Java?

Was anything introduced in Java, in the last editions (1.5/1.6) that facilitate writing strings to files (the Scanner of writing)?

Easier than FileWriter?

It's not a core library in Java but FileUtils in Apache Common IO is a useful class.
FileUtils.writeStringToFile(File file, String data );

I tend to use PrintStream and printf.

I don't think so, I don't imagine what the "Scanner for writing" would be - I don't see a duality there. For simple writing of text, a Writer seems enough.
If you are thinking of something like a formatter, there are good third party libraries, for example, I like Freemarker.

I find I use RandomAccessFile a lot.
RandomAccessFile rafWrite = new RandomAccessFile("<file name>", "<mode, usually rw>");
rafWrite.writeBytes("<text you want to write>");