I have a project where we write a small amount of data to a file every 5 minutes. The idea is to look at how this data changes over a period of hours, days, and weeks.
One of the requirements is to store this data in a secure format. We already have an encryption scheme for sending this data across a network as a byte[] array via DataI/O streams.
The question I have is this, is there a way to write encrypted byte[] arrays to a text file in such a way that I can read them back out? My biggest problem at the moment is that I'm reading Strings from the files, which messes up the byte[] arrays.
Any thoughts or pointers on where to go?
What you need to do is take your data and put it into a byte array. Then once it is in a byte array, you can encrypt it using an encryption algorithm. Then you write it to the file.
When you want to get the original data back, you have to read the byte array from the file, then decrypt the byte array and then you will have your original data. You cannot just read this data as a string because your encryption algorithm will create bytes that cannot be represented as regular chars so your data will get messed up.
Just make sure you read the encrypted data as a byte array and not a string, that is where you are having a problem.
If you want to write multiple byte arrays to a single file, then you should probably do something like this since you are using Java:
writer.print(arr.length);
writer.print(arr);
writer.flush();
Do this for each byte array. Then when you read the byte arrays back:
int length = reader.readInt();
byte[] bytes = new byte[length];
// fill array
This way the file can be structured like this:
[length of following array][array][length of second array][second array]
You will be able to put all of the byte arrays back to back, and since each array starts with the length of the array, you will know how much data needs to be put into each array.
See How to append to AES encrypted file for an example of an AES+CBC Java example which allows opening an already encrypted file and appending more encrypted data to in, while not requiring any special handling when decrypting it since it looks just like it would if the entire file had been encrypted just once.
Related
I am trying to read a .wav file, convert it into double array and FFT the array.
I have the recorded .wav file in storage but I have no idea how I can read the file and use the data.
I am a beginner in application development so it would be nice if you could lead me through step by step as well as show some sample code.
Appreciate your help.
I can't give you a full code since it's a long solution and I have other things to do. I can give you hints.
First, check this link as reference.
As you see, a .wav or a WAVE file does not only contain the audio samples but it also contains other metadata that describes the contents of the file. To correctly read the audio samples, you'll need the values of these metadata.
To do this, first instantiate a FileInputstream. You will need a File object representing your .wav file to do that.
Next, you'll need to read each field from top to bottom. As you see from the illustration, the number of bytes for each field is indicated. Use the following code when reading a field.
byte[] bytes = new byte[numOfBytes];
fileInputStream.read(bytes, 0, bytes.length);
// The value of current field is now stored in the bytes array
After each read, the fileInputStream will automatically point to the next field. Just repeat the above code until you have reached the start of the audio samples data.
To interpret the obtained values, you'll have to carefully read the description of each field.
To convert field from byte array to ASCII or String, use:
String value = new String(bytes);
To convert field from byte array to a number, use:
ByteBuffer buffer = ByteBuffer.allocate(numOfBytes);
buffer.order(BIG_ENDIAN); // Check the illustration. If it says little endian, use
// LITTLE_ENDIAN
buffer.put(bytes);
buffer.rewind();
If field consists of two bytes:
Short value = buffer.getShort();
If field consists of 4 bytes:
Int value = buffer.getInt();
To read the samples, just continue what you're doing above. But you'll also have to consider the number of bits per sample as given by the BitsPerSample field. For 8 bits, read 1 byte. For 16 bits, read 2 bytes, and so on and so forth. Just repeat to get each sample until you reach the end of file.
To check the end of file, get the returned value from read and check if it is -1:
int read = fileInputSream.read(bytes, 0, bytes.length);
// If read equals -1 then end of file
I have a 2D dynamic array containing integers and strings that I need to send to the server. The array comprises of 100,000 columns. One possibility is to copy the array into a file and send it to the server and the server can parse it. Whereas, the other possibility is to do serialisation. I am doing socket programming for the first time and the solution that I am looking for should be optimal. Considering the size of the array, I dont think converting each element into bytes would be good. Please could you give me a slight hint or guide me on how I should proceed.
Serializing the data to bytes for streaming over a socket, or serializing the data to a file, and sending the file is essentially the same thing. If you serialize to a file you can fall back on an established protocol like FTP, which will be easier for a developer new to sockets.
Again however, realize they're essentially the same problem. Breaking down a programmatic object into bytes for writing to an output stream.
Edit:
Depending on the content of your strings, using the zip compression algorithm is a good bet. The more repeated content in your strings, the better compression ratio you'll get. Here's just an example file format:
Number of ints to follow. (As you've not given the max ROWS in your 2D array I can't say how many bytes this will take. We'll assume its 8 bytes.)
All ints. This will take 3*NUM_INTS bytes.
Zip compressed strings, separated by a space.
I'm making a lot of assumptions here because you've not been specific enough in your question, or any replies you've given.
I would convert it to json or xml and send that. Is not the most efficient solutions in terms of speed, but is the most portable.
A simple approach would be to use a DataOutputStream and wrap it around the socket you are using: DataOutputStream dOut = new DataOutputStream(socket.getOuputStream()).
You then could just establish your own simple binary protocol like this (it's for an 1D array, but it can be easily extended to work for 2D arrays as well):
//first write the amount of elements in the array
dOut.write(array.length());
//now write every element
for (Object obj : array) {
if (obj instanceof Integer) {
dOut.writeByte(0); //0 = Integer
dOut.writeInt((Integer) obj);
} else
if (obj instanceof String) {
dOut.writeByte(1); //1 = String
dOut.writeUTF((String) obj); //UTF-8 encoded String
}
// ...
}
If you want to optimize for data size then you could also use a GZIPOutputStream in between the socket's OutputStream and your DataOutputStream like this: new DataOutputStream(new GZIPOutputStream(socket.getOutputStream())). Of course you have to decompress it on server side for this to work.
I'm writing a program which takes in a byte array of potentially millions of bytes, reads each one from a ByteArrayInputStream, and if the byte is not "printable" (ascii 32-126), that byte is encoded in a certain way and written to a ByteArrayOutputStream instance; if the byte is "printable" it is directly written to that same ByteArrayOutputStream instance.
So from a broader view I am taking in a byte array, and getting back a similar byte array except certain characters have been encoded.
My question is: would it be faster to write my data out to a file or to continuously be writing to this OutputStream?
It will be faster to write the data to your output stream. Writing to a file will involve disk access, which is slower than access to the RAM where the byte array inside the ByteArrayOutputStream lives.
However, if you eventually want to write your byte array out to some other place (say a file) then the intermediate step of the ByteArrayOutputStream is unnecessary and you should just write straight to the end destination e.g. FileOutputStream.
As tilte,
I tried to capture the file size info which is saved in 4 bytes of data from bitmap's file code,
but if I use byte[] to save it and any byte of file exceeds 127, it'd be misjudged as negative values,
in this case how could we correct these kind of values?
My book just simply plus 256 to it, but could it fixedly be -128?
if not, then the result still isn't correct.
I know we can just use int[] or larger array for it, just wanna know how to deal with these kind of problems!
Thanks a lot for helping!!!
I have a multi-threaded client-server application that uses Vector<String> as a queue of messages to send.
I need, however, to send a file using this application. In C++ I would not really worry, but in Java I'm a little confused when converting anything to string.
Java has 2 byte characters. When you see Java string in HEX, it's usually like:
00XX 00XX 00XX 00XX
Unless some Unicode characters are present.
Java also uses Big endian.
These facts make me unsure, whether - and eventually how - to add the file into the queue. Preferred format of the file would be:
-- Headers --
2 bytes Size of the block (excluding header, which means first four bytes)
2 bytes Data type (text message/file)
-- End of headers --
2 bytes Internal file ID (to avoid referring by filenames)
2 bytes Length of filename
X bytes Filename
X bytes Data
You can see I'm already using 2 bytes for all numbers to avoid some horrible operations required when getting 2 numbers out of one char.
But I have really no idea how to add the file data correctly. For numbers, I assume this would do:
StringBuilder packetData = new StringBuilder();
packetData.append((char) packetSize);
packetData.append((char) PacketType.BINARY.ordinal()); //Just convert enum constant to number
But file is really a problem. If I have also described anything wrongly regarding the Java data types please correct me - I'm a beginner.
Does it have to send only Strings? I think if it does then you really need to encode it using base64 or similar. The best approach overall would probably be to send it as raw bytes. Depending on how difficult it would be to refactor your code to support byte arrays instead of just Strings, that may be worth doing.
To answer your String question I just saw pop up in the comments, there's a getBytes method on a String.
For the socket question, see:
Java sending and receiving file (byte[]) over sockets