I have a block of static data that I need to organize into an array containing hash maps. Specifically, I want to have a static object in my app that contains the time zone information like this: https://gist.github.com/pamelafox/986163
Seeing how clean the definition looks like in Python, and knowing how a similarly clean definition can be created with some of the other languages I know, I was hoping there is a cleaner approach to it in Java then just running map.put(...) repeatedly. I have seen this question: How to give the static value to HashMap? but what wondering if there is a better way to do it?
One solution would be to store the data as a normal string in whatever format you can think of and then convert the string representation into the map (static, non-static or as a one-time initialized instance).
An improvement of this method would be to store the data in a file and load it (can be included in .jar package, when you use jar). This solution would have the advantage that data can be easily updated.
Related
In Python i have a class with some string attributes and a function that returns an object of this class with atttributes set (sometimes can return an array of objects).
Theres any way to get this return in Java? Where i can see the strings of the object?
I Tried Jython but couldnt make it work!
Use json.dump function in Python to serialize your object into json format. Then use something like json.org library in Java to parse this object into Java object, some example over here.
Mind that not every object might be serializable, in general data structures like dictionaries or lists are easily serializable, from your description it seems like you want to move an instance of an object from one program into another, which is not possible to be done automatically and requires human work in rewriting the code as instances of classes contain not only data but also functions (methods).
Good luck!
I need to parse untrusted Java serialized objects. The data is given to me as a byte array (written at some point by ObjectOutputStream).
I do not want to simply call ObjectInputStream.readObject() and/or load the actual object. I am looking for a way to safely parse the bytes and grab field names & values.
--
Here's a little summary of my attempt so far, after taking a look at the ObjectInputStream procedure for deserializing objects.
I have tried to extract field types/names (as unicode strings) recursively based on expected stream constants. I end up with a list of field names whose values should appear in the byte array in order. I am uneasy about this approach because it is probably buggy. Especially accommodating for what seems to be individual serialization protocols followed by HashMap, ArrayList, etc. But it might work, if I can figure out a way to read the bytes that represent field values:
I can try to read and store primitives based on size/offset, but when I encounter my first object, it gets a bit more complicated -- there is no clear way to distinguish between which bytes are associated with which values anymore (without actually loading the object in the way that ObjectInputStream probably does?).
--
Can anyone suggest either a potential solution that I'm obviously looking past, or a trusted library that can help parse the serialized data without loading objects?
Thank you for reading, and for all comments/suggestions!!! I apologize if something is unclear and I would be happy to clarify if you bear with me.
You can't do this in principle. Any Java class can take over its own Serialization and write arbitrary data to the stream that only it knows how to parse and reconstruct, via code that is only invoked during deserialization.
This is my first question to StackOverflow. Please let me know if the question is not clear and need any more details.
I have a class which has three attributes like this:
class SampleClass {
long [] field1;
float[] field2;
float[] field3;
}
A huge SampleClass object is built(with about a billion entries for each array). This object is serialized in one host and the serialized file is uploaded to another machine. Now I want to deserialize only a portion of the file so that I can get a smaller SampleClass object with about 10 indices filled for each field and not the complete object. Because this machine does not have enough capacity to load such a huge object in memory. Is this possible?
The object is serialized using JAVA's writeObject method and it is done by a different utility and so I have no control over it. Thanks in advance.
Forget using the Java serialization API - it's only designed to deserialize everything. If you have no control over how the serialized file is generated, then you should consider parsing the serialized file yourself and extracting the necessary parts - it's not really that hard.
The Java serialization format is well-documented (see e.g. official docs, informative article), and tools exist to parse the format (e.g. Serialysis, jdeserialize) though it isn't particularly hard to write your own tool based on the format spec.
Once you can parse the serialized data, you can simply extract what you need and skip over what you don't need.
Your best bet is to actually serialize only the portion you need, given that you cannot control/override serialization itself. On the machine which serialized entire file and is able to deserialize it:
1) load entire file into object
2) create new object of SampleClass
3) copy elements from required region in each array to blank SampleClass object
4) serialize this smaller version
If it helps any, fields can be made transient so they will not be serialized.
Still, it looks to me that this object should be in database:
It does not fit virtual memory
only portion of it is required at given time.
So you could use hard disk to store it and queries to get required portions.
Java program takes a long list of inputs(parameters), churns a bit and spits some output.
I need a way to organize these parameters in a sane way so in the input txt file I want to write them like this:
parameter1 = 12
parameter2 = 10
strategy1.parameter1 = "goofy"
strategy2.parameter4 = 100.0
Then read this txt file, turn it into a Java object I can pass around to objects when I instantiate them.
I now pyqtgraph has ParameterTree which is handy to use; is there something similar in Java? I am sure others must have had the same need so I don't want to reinvent the wheel.
(other tree structures would also be fine, of course, I just wanted something easy to read)
One way is to turn input.txt into input.json:
{
"parameter1": 12,
"parameter2": 10,
"strategy1": {
"parameter1": "goofy"
},
"strategy2": {
"parameter4": 100.0
}
}
Then use Jackson to deserialize input.json into one of these:
A Map<String, Object> instance, which you could navigate in depth to get all your parameters
An instance of some class of your own that mimics input.json's structure, where your parameters would reside
A JsonNode instance that would be the root of the tree
(1) has the advantage that it's easy and you don't have to create any class to read the parameters, however you'd need to traverse the map, downcast the values you get from it, and you'd need to know the keys in advance (keys match json object's attribute names).
(2) has the advantage that everything would be correctly typed upon deserialization; no need to downcast anything, since every type would be a field of your own classes which represent the structure of the parameters. However, if the structure of your input.json file changed, you would need to change the structure of your classes as well.
(3) is the most flexible of all, and I believe it's the option that is closest to what you have in mind, nonetheless is the most tedious to work with, since it's too low-level. Please refer to this article for further details.
Currently I have a class setup to be processed as an autobean:
public interface Asset extends Hit {
String getGuid();
String getHitType();
Map<String,Serializable> getMetadata();
}
I tried using Object instead of Serializable:
Map<String,Object> getMetadata()
but this seems to blow up when trying to access data (because it's not 'reified').
The Metadata map may contain other maps, strings, ints, etc. How do I retrieve data from an inner map of that metadata object?
Currently, if I call asset.getMetadata().get("title"); this returns a SerializableAutoBean and performing toString() or String.valueOf(obj) on that object returns the in memory object information and not the actually string value.
Can an AutoBean object be this dynamic, or do you specifically have to define every field?
AutoBeans aren't "dynamic" in the Java generics or RTTI sense.
In GWT, all types have to be known at compile time for anything which is auto-generated (which includes AutoBeans). This places restrictions on your designs which don't allow you to take full advantage of Java's language features (specifically, generics and other RTTI features). So, AutoBeans are not dynamic in the RTTI or Java generic sense. However, AutoBeans are simply a low-level way of wrapping your data, and you still have access to the data by using Splittables!
As stated in the previous comments, you can use Splittables for the parts of your JSON object whose type is not known at serialization/decode time. Sure, it would be nice to have everything happen at once, but nothing is stopping you from performing some post-processing on your data objects to get them into your desired state.
A really good way for someone to "Grok" what is going on with AutoBeans (and anything else which is autogenerated) is to look at the resulting generated code. The default location for maven is: ${project.build.directory}/.generated.
If you look in there after you've compiled, you should find the code which the GWT compiler produces for your AutoBeans.