Develop Reference

Dynamically creating an identical and independent copy of a neural network. JAVA

Context
I am making a simulation of behavioral evolution. For this I am using neural networks to simulate behavior. There are literally thousands of these neural networks interacting with each other. At the end of any given generation the strongest behavior is copied onto its neighbors. When this happens I need to create an identical yet independent version of the stronger neural network onto the new one; thus replacing it.
The Problem
I have looked into deep cloning, it works but it is copying node references, not creating a new identical instance. The difficulty comes from the system structure. I can't see how to ameliorate it though.
Speed is a factor here as well. I need this to run millions of iterations a week.
Any help would be greatly appreciated.
System structure
The cell
public class Cell_NN Extends Cell
{
private Network network;
//Methods
}
The network
public class Network implements Cloneable, Serializable
{
private ArrayList<ArrayList<Node>> net;
private ArrayList<Node> layer;
//Methods
}
The nodes
public class Node implements Cloneable, Serializable
{
private ArrayList<Node> nextNodes;
private ArrayList<Float> weights;
//Methods
}
The deep clone (which I ripped off someone from stackOverflow)
public Network deepClone()
{
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(this);
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
ObjectInputStream ois = new ObjectInputStream(bais);
Network network = (Network) ois.readObject();
return (Network) ois.readObject();
} catch (IOException e) {
return null;
} catch (ClassNotFoundException e) {
return null;
}
}
I am unsure whether this is allowed but here is my github in case you would like more information: Napier40124399. Project is called HonorsMain_v2 and is public.

I think your code contains just a small bug:
Network network = (Network) ois.readObject();
return (Network) ois.readObject();
Here you read the network twice from the ObjectInputStream, and return the second instance (which should be null). If you remove the first line, it should work.
The object is deep-cloned, and all references in the new Network instance are correctly wired to each other, not the original objects.
However, performance-wise this serialization / deserialization is pretty expensive. I recommend finding a structure which is faster to process if performance is of the essence.
For instance, two flat arrays for each network, one with the weights (as primitive floats or even ints) and one with int indices to the next nodes would be at least an order of magnitude faster to copy, using System.arraycopy().

This part of the deep clone code looks odd to me:
Network network = (Network) ois.readObject();
return (Network) ois.readObject();
Reading the object twice, and only returning the 2nd one... I would expect an exception here.

Serializing arbitrary Java objects

I am currently in need to serialize arbitrary Java objects since I would like to use the Hash as a key for a hash table. After I read various warnings that the default hashCode creates collisions way to often, I wanted to switch to hashing via MessageDigest to use alternative algorithms (e.g. SHA1, ...) that are said to allow more entries without collisions. [As a sidenote: I am aware that even here collisions can occur early on, yet I want to increase the likelihood to remain collision free.]
To achieve this I tried a method proposed in this StackOverflow post. It uses the following code to obtain a byte[] necessary for MessageDigest:
public static byte[] convertToHashableByteArray(Object obj) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
byte[] byteOutput = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(obj);
byteOutput = bos.toByteArray();
} catch (IOException io) {
io.printStackTrace();
} finally {
try {
if(out != null) { out.close(); }
} catch(IOException io) {
io.printStackTrace();
}
try {
bos.close();
} catch(IOException io) {
io.printStackTrace();
}
}
return byteOutput;
}
This, however, causes the problem that only objects implementing the serializable interface will be serialized/converted into a byte[]. To circumvent this issue I applied toString() to the given obj in the catch clause to enforce getting a byte[] in all cases:
public static byte[] convertToHashableByteArray(Object obj) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
byte[] byteOutput = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(obj);
byteOutput = bos.toByteArray();
} catch (IOException io) {
String stringed = obj.toString();
byteOutput = stringed.getBytes();
} finally {
try {
if(out != null) { out.close(); }
} catch(IOException io) {
io.printStackTrace();
}
try {
bos.close();
} catch(IOException io) {
io.printStackTrace();
}
}
return byteOutput;
}
However, this still feels utterly wrong for me. So my question is, whether there is a better alternative to convert arbitrary objects to byte[] to be able to compute hashes. Preferably a solution that works without using additional libraries or one using well established ones like Apache Commons.
(Beside that I am also open for other approaches to obtain SHA1/SHA512 hashes of arbitrary Java objects.)

Perhaps you can use UUIDs for your objects as immutable unique identifiers?

There are so many things wrong here...
You should have proper key classes with equals and hashCode implemented, instead of using random objects.
Serialization performance overhead can easily mean that such map will be slower than even trivial iteration search.
Default hashcode should not be used in most cases, as it might be different for objects which are 'equal' from business point of view. You should reimplement hashcode together with equals (which comes back to point 1). Whenever it has collisions due to pointer aliasing is irrelevant if it won't work poperly
Way overcomplicated method of closing in-memory streams. Just close them one after another, it is not external resource - if it fails, just let it fail, you don't need to close everything 100% in case of failures. You can also use one of closeable utilities (or try/catch with resources) to avoid some overhead
You don't need complicated digest of that byte array - use Arrays.hashCode, it WILL be good enough for your use cases (remember - don't do it anyway, point 1)
If you are still reading and still not willing to implement point 1, go back to point 1. And again. And again.
And to finally answer you question, use hessian serialization.
http://hessian.caucho.com/doc/hessian-overview.xtp
It is very similar to java one, just faster, shorter output and allows serializing objects which do not implement Serializable interface (at the risk of messing things up, you need to set special flag to allow that).

If you want to serialize a given object, i suggest you change your méthod like this :
public static byte[] convertToHashableByteArray(Serializable obj){
..........
..........
}

Serializable Errors with Java Object

Edit, Here's how i solved using the comments
So after trying different ways of serializing and looking through my code, I finally found out that each object drawn in the renderer contains FloatBuffers. I created a capsule class thanks to Ted Hopp. Then I tried returning the float representation of the FloatBuffers using .array(), which you can't do. My guess is because these are running on threads. So using a suggestion from Learn OpenGL ES to use get, i instead did
public float[] getVertexBuffer()
{
float[] local = new float[vertexBuffer.capacity()];
vertexBuffer.get(local);
return local;
}
Which does work and returns the float[].
Then i store them all in a capsule object for each mGrid object i created
Encapsulate capsule = new Encapsulate(values);
for(int i = 0; i < values[0]; i++)
{
for(int j = 0; j < values[1]; j++)
{
capsule.storeVertex(i,j,mRenderer.mGrid[i*values[1] + j].getVertexBuffer());
capsule.storeColors(i,j,mRenderer.mGrid[i*values[1] + j].getmColors());
capsule.storePillar(i,j,mRenderer.mGrid[i*values[1] + j].getPillarPositions());
}
}
Which I can then ultimately save because it's serializable. Thank you all
PROBLEM DESCRIPTION
So i'm trying to save a GLSurfaceView object, whose class is denoted as
class GLWorld extends GLSurfaceView implements Serializable
Now I'm sure as i do the saving correctly.
public void saveSimulation()
{
String fileName = "Test Save";
try {
FileOutputStream fos = openFileOutput(fileName, Context.MODE_PRIVATE);
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(mGLView);
Log.d("Save","Successfully Written");
oos.close();
fos.close();
} catch (FileNotFoundException e) {
Log.d("Save","File not found exception");
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
Log.d("Save","IO exception");
// TODO Auto-generated catch block
e.printStackTrace();
}
finish();
}
But i'm getting an error i have no clue how to fix. I've spent hours looking around but haven't found anything like it.
09-16 17:36:50.639: W/System.err(2996): java.io.NotSerializableException: java.nio.FloatToByteBufferAdapter
Along with many more system err lines below that, which i believe stem from this one error.
My GLWorld creates a renderer object in it which has different objects with floatbuffers in it which store vertex and color data. I can't figure out what to do to get past this error, or why those float buffers are throwing an error. Everything runs smoothly except actually trying to save this GLWorld object and it's driving me insane.

Just declaring that a class implements Serializable is not enough to successfully serialize objects of that class. The default implementation requires that every field of the class be serializable. In your case, there's a field of type FloatToByteBufferAdapter that isn't serializable (there may be more).
You can define your own serialization mechanism to serialize only what you need. The details can be found in the Serializable docs. Be aware that by subclassing GLSurfaceView, it is unlikely you will be able to successfully deserialize this class, even if you write the correct support methods. For one thing, GLSurfaceView does not have a default (no-arg) constructor, which is a requirement of Java's serialization mechanism. Also, many objects simply cannot be serialized (e.g., streams).
I suggest that you encapsulate the data you want to serialize in a helper class and limit the serialization/deserialization to those data.

Gotta assume that something within the mGLView inheritance contains FloatTOByteBufferAdapter, which isn't serializable.

Save game to file

I'm working on a primitive RPG, and this class (supposedly) contains all the necessary data:
public class CPU implements Serializable{
private Map<String, Location> locations;
private Map<String, Location> places;
private Map<String, NPC> npcs;
private Game game;
private Player player;
private NPC currentNPC;
public CPU(){
}
(I didn't include the methods, but I think those are irrelevant right now...)
The class "Game" also contains the Player and the CPU as variables, but its constructor isn't the one to actually create them (those are created in the main() method, then added to the classes). This method is the one that's supposed to save the CPU class to a file, so that I can read all the data from it later:
public void SaveGame(String s){
String sav = s;
sav.concat(".dat");
try {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(sav));
oos.writeObject(cpu);
oos.close();
} catch(Exception ex) {
ex.printStackTrace();
}
}
And this is the method to load it from the file:
public void Load(String s){
if(s.contains(".dat")){
try {
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(s));
cpu = (CPU)ois.readObject();
ois.close();
} catch(Exception ex) {
ex.printStackTrace();
}
}
}
My question is basically: will this work? Am I able to simply serialize the CPU class and save it to a file, then read it back and be able to recover all the data from it (i.e the Player data)?
If I remember correctly, in Java "=" doesn't mean that the object on the right side will be copied, so my other problem is: when the method "Load" finishes, will the "cpu" (variable of the "Game"-class) still contain the CPU that I loaded from the file, and will I be able to read data from it?

Actually, it will work only if the original file name that you pass to the SaveGame() method contains ".dat".
Indeed, the reading method checks that condition, and the SaveGame() method (which should be named saveGame() to respect the Java naming conventions) doesn't append .dat to the file name as you think. Indeed, Strings are immutable, and the concat() method returns a new String, but doesn't modify the String it's called on. The code should be
String sav = s.concat(".dat");
You should also stop ignoring exceptions like you're doing, and you should always close the streams in a finally block. If you're under Java 7, use the try-with-resources construct.

If your program has permissions to write files, and all of the field classes (NPC, Game, Player. Location) are serializable as well, then it will work.
The cpu field of the Game class will contain the CPU loaded from the file, if no exception is thrown when reading the object.

java string scramble

I am currently working on a program that takes user data (name, address, phone). I want to make the data unreadable when I save it. Since it won't be a released program, I don't want to use a cipher that is too complicated, just a simple and quick scramble/unscramble algorithm.

How about defining a class (let's say Foo) and create instances of that class those which can hold the data, then put the objects into a list. After that, save the list object as a binary file. Something like this:
public static void saveObject(List<Foo> obj, String filePath)
{
OutputStream os = null;
try
{
os = new ObjectOutputStream(new FileOutputStream(filePath));
os.writeObject(obj);
}
catch(Exception ex){}
finally
{
os.close();
}
}
Then you can load it like:
public static List<Foo> loadObject(String filePath)
{
List<Foo> obj = null;
InputStream is = null;
try
{
is = new ObjectInputStream(new FileInputStream(filePath));
obj = (List<Foo>) is.readObject();
}
catch(Exception ex){}
finally
{
is.close();
}
return obj;
}
Note that you need to implement the Serializable interface to use this object serialization.

you just need to understand that its a matter of strength of your algorithm, because according to your post, you do need some way of encryption/decryption.
You can think about a lot of ways to "hide" your data but this "defense" can be broken in a relatively easy way... So its up to you to decide.
The easiest implementations are (naive):
- substitute each letter in your string with some number.
- play with ascii values of your letters in string.
Again the real question here is whether it is good enough???
From the point of view of "security" - definitely not.