I am currently working on a program that takes user data (name, address, phone). I want to make the data unreadable when I save it. Since it won't be a released program, I don't want to use a cipher that is too complicated, just a simple and quick scramble/unscramble algorithm.
How about defining a class (let's say Foo) and create instances of that class those which can hold the data, then put the objects into a list. After that, save the list object as a binary file. Something like this:
public static void saveObject(List<Foo> obj, String filePath)
{
OutputStream os = null;
try
{
os = new ObjectOutputStream(new FileOutputStream(filePath));
os.writeObject(obj);
}
catch(Exception ex){}
finally
{
os.close();
}
}
Then you can load it like:
public static List<Foo> loadObject(String filePath)
{
List<Foo> obj = null;
InputStream is = null;
try
{
is = new ObjectInputStream(new FileInputStream(filePath));
obj = (List<Foo>) is.readObject();
}
catch(Exception ex){}
finally
{
is.close();
}
return obj;
}
Note that you need to implement the Serializable interface to use this object serialization.
you just need to understand that its a matter of strength of your algorithm, because according to your post, you do need some way of encryption/decryption.
You can think about a lot of ways to "hide" your data but this "defense" can be broken in a relatively easy way... So its up to you to decide.
The easiest implementations are (naive):
- substitute each letter in your string with some number.
- play with ascii values of your letters in string.
Again the real question here is whether it is good enough???
From the point of view of "security" - definitely not.
Related
In this opencv example, the Mat object has a nativeObj field, returning a long that represents the address of the object (i.e 140398889556640). Because the size of the data within the object is known, I wish to access the contents of the Mat object directly, returning a byte buffer.
What is the best way to do so?
You can wrap the address with a DirectByteBuffer or use Unsafe.
While you can do this, you probably shouldn't. I would explore all other options first.
// Warning: only do this if there is no better option
public static void main(String[] args) {
ByteBuffer bb = ByteBuffer.allocateDirect(128);
long addr = ((DirectBuffer) bb).address();
ByteBuffer bb2 = wrapAddress(addr, bb.capacity());
bb.putLong(0, 0x12345678);
System.out.println(Long.toHexString(bb2.getLong(0)));
}
static final Field address, capacity;
static {
try {
address = Buffer.class.getDeclaredField("address");
address.setAccessible(true);
capacity = Buffer.class.getDeclaredField("capacity");
capacity.setAccessible(true);
} catch (NoSuchFieldException e) {
throw new AssertionError(e);
}
}
public static ByteBuffer wrapAddress(long addr, int length) {
ByteBuffer bb = ByteBuffer.allocateDirect(0).order(ByteOrder.nativeOrder());
try {
address.setLong(bb, addr);
capacity.setInt(bb, length);
bb.clear();
} catch (IllegalAccessException e) {
throw new AssertionError(e);
}
return bb;
}
If you don't want to use Unsafe and want something that works without warnings in Java 9 and is actually portable across JVMs you can use JNIs NewDirectByteBuffer. This is API and guaranteed to work.
You will need to write some C (or C++) code however and ship a native library with your code.
There is a tiny framework called "nalloc" which is designed to help developer with memory/pointers manipulations, it could be useful for whatever purposes you are looking for direct memory address accessing.
Also it brings you ability to write your Java program in a C-style, doing memory stuff manually.
Check it out: https://github.com/alaisi/nalloc
I am currently in need to serialize arbitrary Java objects since I would like to use the Hash as a key for a hash table. After I read various warnings that the default hashCode creates collisions way to often, I wanted to switch to hashing via MessageDigest to use alternative algorithms (e.g. SHA1, ...) that are said to allow more entries without collisions. [As a sidenote: I am aware that even here collisions can occur early on, yet I want to increase the likelihood to remain collision free.]
To achieve this I tried a method proposed in this StackOverflow post. It uses the following code to obtain a byte[] necessary for MessageDigest:
public static byte[] convertToHashableByteArray(Object obj) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
byte[] byteOutput = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(obj);
byteOutput = bos.toByteArray();
} catch (IOException io) {
io.printStackTrace();
} finally {
try {
if(out != null) { out.close(); }
} catch(IOException io) {
io.printStackTrace();
}
try {
bos.close();
} catch(IOException io) {
io.printStackTrace();
}
}
return byteOutput;
}
This, however, causes the problem that only objects implementing the serializable interface will be serialized/converted into a byte[]. To circumvent this issue I applied toString() to the given obj in the catch clause to enforce getting a byte[] in all cases:
public static byte[] convertToHashableByteArray(Object obj) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
byte[] byteOutput = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(obj);
byteOutput = bos.toByteArray();
} catch (IOException io) {
String stringed = obj.toString();
byteOutput = stringed.getBytes();
} finally {
try {
if(out != null) { out.close(); }
} catch(IOException io) {
io.printStackTrace();
}
try {
bos.close();
} catch(IOException io) {
io.printStackTrace();
}
}
return byteOutput;
}
However, this still feels utterly wrong for me. So my question is, whether there is a better alternative to convert arbitrary objects to byte[] to be able to compute hashes. Preferably a solution that works without using additional libraries or one using well established ones like Apache Commons.
(Beside that I am also open for other approaches to obtain SHA1/SHA512 hashes of arbitrary Java objects.)
Perhaps you can use UUIDs for your objects as immutable unique identifiers?
There are so many things wrong here...
You should have proper key classes with equals and hashCode implemented, instead of using random objects.
Serialization performance overhead can easily mean that such map will be slower than even trivial iteration search.
Default hashcode should not be used in most cases, as it might be different for objects which are 'equal' from business point of view. You should reimplement hashcode together with equals (which comes back to point 1). Whenever it has collisions due to pointer aliasing is irrelevant if it won't work poperly
Way overcomplicated method of closing in-memory streams. Just close them one after another, it is not external resource - if it fails, just let it fail, you don't need to close everything 100% in case of failures. You can also use one of closeable utilities (or try/catch with resources) to avoid some overhead
You don't need complicated digest of that byte array - use Arrays.hashCode, it WILL be good enough for your use cases (remember - don't do it anyway, point 1)
If you are still reading and still not willing to implement point 1, go back to point 1. And again. And again.
And to finally answer you question, use hessian serialization.
http://hessian.caucho.com/doc/hessian-overview.xtp
It is very similar to java one, just faster, shorter output and allows serializing objects which do not implement Serializable interface (at the risk of messing things up, you need to set special flag to allow that).
If you want to serialize a given object, i suggest you change your méthod like this :
public static byte[] convertToHashableByteArray(Serializable obj){
..........
..........
}
Motivation:
To aid in remote debugging (Java), it's useful to be able to request remote servers to send over arbitrary objects to my local machine for inspection. However, this means that the remote server must be able to serialize an arbitrary java object that is not known in advance at runtime.
In particular, I would like to be able to serialize even those objects which don't implement Serializable. I stumbled upon JBossSerialization which claimed with that with JBossSerialization...
...You can serialize classes that are not implementing Serializable
Great! And even better, I managed to find the code that supposedly demonstrates how to do this.
Problem
So pinching the code from schabell.org, I wrote a quick test to check that I could serialize and deserialize without problems:
import org.jboss.serial.io.JBossObjectInputStream;
import org.jboss.serial.io.JBossObjectOutputStream;
import java.io.*;
class MyObj { // Test class which doesn't implement Serializable
public int x;
MyObj(int x) {this.x = x;}
}
public class SerializationTest {
public static void main(String[] args) {
MyObj obj = new MyObj(1);
byte[] byteArray = getByteArrayFromObject(obj); // Try to serialize
MyObj result = (MyObj) getObjectFromByteArray(byteArray); // Try to deserialize
System.out.println(result.x);
}
// Code that I pinched from website below (http://www.schabell.org/2009/03/jboss-serialization-simple-example.html):
public static Object getObjectFromByteArray(byte[] bytes) {
Object result = null;
try {
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
ObjectInputStream ois = new JBossObjectInputStream(bais);
result = ois.readObject(); // ERROR HERE!!!
ois.close();
} catch (IOException ioEx) {
ioEx.printStackTrace();
} catch (ClassNotFoundException cnfEx) {
cnfEx.printStackTrace();
}
return result;
}
public static byte[] getByteArrayFromObject(Object obj) {
byte[] result = null;
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new JBossObjectOutputStream(baos);
oos.writeObject(obj);
oos.flush();
oos.close();
baos.close();
result = baos.toByteArray();
} catch (IOException ioEx) {
ioEx.printStackTrace();
}
return result;
}
}
Problem is that the test failed. Debugging indicated that I could only serialize, but not deserialize. The call to ois.readObject() on line 26 is the culprit and gives as SerializationException:
org.jboss.serial.exception.SerializationException: Could not create instance of MyObj - MyObj
at org.jboss.serial.classmetamodel.ClassMetaData.newInstance(ClassMetaData.java:342)
at org.jboss.serial.persister.RegularObjectPersister.readData(RegularObjectPersister.java:239)
at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.readObjectDescriptionFromStreaming(ObjectDescriptorFactory.java:412)
at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.objectFromDescription(ObjectDescriptorFactory.java:82)
at org.jboss.serial.objectmetamodel.DataContainer$DataContainerDirectInput.readObject(DataContainer.java:643)
at org.jboss.serial.io.JBossObjectInputStream.readObjectOverride(JBossObjectInputStream.java:163)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
at SerializationTest.getObjectFromByteArray(SerializationTest.java:44)
at SerializationTest.main(SerializationTest.java:15)
Caused by: java.lang.InstantiationException: MyObj
at java.lang.Class.newInstance(Class.java:359)
at org.jboss.serial.classmetamodel.ClassMetaData.newInstance(ClassMetaData.java:334)
... 8 more
Does anyone know what's going wrong here and how I can get round this?
Or indeed if JBossSerialization isn't the right tool for this, what is?
Edit:
As #Dima points out, the SerializationException is caused by a lack of a public default constructor of the MyObj class. However, adding a default constructor to MyObj isn't an option as I'd like to be able to serialize arbitrary objects, including those without a default constructor.
Well, it is actually impossible to do what you want in a way, that would be both safe and universal.
You can take a look at Kryo, as someone suggested in comments as well. It does have a way to instantiate objects without invoking a constructor, but it is off by default and there is a good reason for it.
Consider this for example:
public class CanonicalObject {
public static HashMap<String,CannicalObject> canons = new HahMap<~>();
public String name;
private CanonicalObject(String name) {
this.name = name;
canons.put(name, this);
}
public static synchronized CanonicalObject getCanonicalInstance(String name) {
CanonicalObject co = canon.get(name);
return co == null ? new CanonicalObject(name) : co;
}
}
(This is a "semi-real-life" example, in that there are real uses for this pattern. I am aware of the "memory leak", there are ways to avoid it in real applications, but they are irrelevant to this example, so I am just ignoring that issue for the sake of simplicity).
If you serialize an instance of this object, when you deserialize it on the other end, the whole "canonicalization" part will be skipped, which can cause subtle problems in application, that are really hard to diagnose, such as comparisons like if(canon1 != canon2) fireMissile() resulting in "friendly fire" and, possibly, a WorldWar III.
Note, that the problem here is broader than just a constructor not invoked by deserialization: the canon.put call could very well be put into getCanonicalInstance() instead of the constructor, and that would present the problem even if the constructor was invoked.
This is an illustration of why, as a matter of policy, you should not be serializing objects that are not designed to be serialized. IT can sometimes work, but, when it does not, it results in situations that are really hard to detect, and usually even harder to fix.
This is the object which I'm planning to store in disk,
public class SObject {
/**
*
*/
private String value;
private int occurences;
private String key;
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
public int getOccurences() {
return occurences;
}
public void setOccurences(int occurences) {
this.occurences = occurences;
}
public String getKey() {
return key;
}
public void setKey(String key) {
this.key = key;
}
}
I am thinking of creating a list of objects and Store that list to a new class like below,
public class ToDisk implements Serializable{
List<SObject> sobj;
public List<SObject> getSobj() {
return sobj;
}
public void setSobj(List<SObject> sobj) {
this.sobj = sobj;
}
}
Is this an efficient way with minimal code work to write data? I also find it easier to read the data from the disk by using serialization. Because the size of list could be very high (~10k+), and I should find a balance between ease of code and size. Is there any way to achieve this balance? Please let me know if there are any other methods to store data in a reduced size compared to serialization. I cannot use any external libraries.
BTW You don't need a list wrapper to write many items to a file, but your items so need to be Serializable.
public class SObject {
private String value;
private int occurences;
private String key;
}
to write
List<SObject> list = new ArrayList<>();
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(fileName));
oos.writeObject(list);
oos.close();
to read
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(fileName));
List<SObject> list = (List<SObject>) ois.readObject(); // cast is needed.
ois.close()
to write compressed
ObjectOutputStream oos = new ObjectOutputStream(new DeflaterOutputStream(new FileOutputStream(fileName)));
oos.writeObject(list);
oos.close();
to read compressed
ObjectInputStream ois = new ObjectInputStream(new InflaterInputStream(new FileInputStream(fileName)));
List<SObject> list = (List<SObject>) ois.readObject(); // cast is needed.
ois.close()
Is this an efficient way with minimal code work to write data?
You have to make SObject implement Serializable and that is all. You can't make it shorter.
You can make it more efficient by making it implement Externalizable but this is more work.
I also find it easier to read the data from the disk by using serialization.
Sounds like a good reason to do it.
Because the size of list could be very high (~10k+),
10K sounds small to me unless your strings are really long. I have a test for my serialization/peristsence library and it writes and reads 10,000,000 entries in seconds.
I should find a balance between ease of code and size. Is there any way to achieve this balance?
I suspect ease is all you need worry about. If youw ant to make the file smaller you can compress it by wrapping the file stream with GZIP or deflator/inflator stream. This adds a line of code and can make the file 4x smaller.
Please let me know if there are any other methods to store data in a reduced size compared to serialization. I cannot use any external libraries.
They all require external libraries. You can use XMLEncoder and XMLDecoder, but this will be slower and bigger.
Since I got a question on it; If you are interested in my library Java Chronicle 2.x Here is a result from a test running on my laptop. This time how long it took to write 500 million entries and read them in a second thread (as they were being written) It took just over 12 seconds.
Rate = 41.0 Mmsg/sec for 16 byte messages
The test is multiThreaded() in IndexedChronicleTest
Best way will probably be to Serialize the ArrayList to disk.
Make sure that the contents of the list are serializable, i.e., the objects in the list should implement the Serializable interface.
then
to write to file:
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(fileName));
oos.writeObject(list);
oos.flush();
oos.close();
To read from file:
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(fileName));
List<YourClass> list = ois.readObject();
ois.close()
I'm working on a primitive RPG, and this class (supposedly) contains all the necessary data:
public class CPU implements Serializable{
private Map<String, Location> locations;
private Map<String, Location> places;
private Map<String, NPC> npcs;
private Game game;
private Player player;
private NPC currentNPC;
public CPU(){
}
(I didn't include the methods, but I think those are irrelevant right now...)
The class "Game" also contains the Player and the CPU as variables, but its constructor isn't the one to actually create them (those are created in the main() method, then added to the classes). This method is the one that's supposed to save the CPU class to a file, so that I can read all the data from it later:
public void SaveGame(String s){
String sav = s;
sav.concat(".dat");
try {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(sav));
oos.writeObject(cpu);
oos.close();
} catch(Exception ex) {
ex.printStackTrace();
}
}
And this is the method to load it from the file:
public void Load(String s){
if(s.contains(".dat")){
try {
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(s));
cpu = (CPU)ois.readObject();
ois.close();
} catch(Exception ex) {
ex.printStackTrace();
}
}
}
My question is basically: will this work? Am I able to simply serialize the CPU class and save it to a file, then read it back and be able to recover all the data from it (i.e the Player data)?
If I remember correctly, in Java "=" doesn't mean that the object on the right side will be copied, so my other problem is: when the method "Load" finishes, will the "cpu" (variable of the "Game"-class) still contain the CPU that I loaded from the file, and will I be able to read data from it?
Actually, it will work only if the original file name that you pass to the SaveGame() method contains ".dat".
Indeed, the reading method checks that condition, and the SaveGame() method (which should be named saveGame() to respect the Java naming conventions) doesn't append .dat to the file name as you think. Indeed, Strings are immutable, and the concat() method returns a new String, but doesn't modify the String it's called on. The code should be
String sav = s.concat(".dat");
You should also stop ignoring exceptions like you're doing, and you should always close the streams in a finally block. If you're under Java 7, use the try-with-resources construct.
If your program has permissions to write files, and all of the field classes (NPC, Game, Player. Location) are serializable as well, then it will work.
The cpu field of the Game class will contain the CPU loaded from the file, if no exception is thrown when reading the object.