I have a problem with this method
private static boolean getBlocks(File file1, File file2) throws IOException {
FileChannel channel1 = new FileInputStream(file1).getChannel();
FileChannel channel2 = new FileInputStream(file2).getChannel();
int SIZE = (int) Math.min((8192), channel1.size());
int point = 0;
MappedByteBuffer buffer1 = channel1.map(FileChannel.MapMode.READ_ONLY, 0, channel1.size());
MappedByteBuffer buffer2 = channel2.map(FileChannel.MapMode.READ_ONLY, 0, channel2.size());
byte [] bytes1 = new byte[SIZE];
byte [] bytes2 = new byte[SIZE];
while (point < channel1.size() - SIZE) {
buffer1.get(bytes1, point, SIZE);
buffer2.get(bytes2, point, SIZE);
if (!compareBlocks(bytes1, bytes2)) {
return false;
}
point += SIZE;
}
return true;
}
private static boolean compareBlocks (byte[] bytes1, byte[] bytes2) {
for (int i = 0; i < bytes1.length; i++) {
if (bytes1[i] != bytes2[i]) {
return false;
}
}
return true;
}
In a result I caught IndexOutOfBoundsException in while loop.
How can I get around this problem and compare two files by blocks?
Yeah... it has to crap.
You create a byte array with 'SIZE' length and access it's position with point var which increments with 'SIZE' vallue.
For example:
int SIZE = 10;
int point = 0;
while( point < channel.size() - SIZE ){
buffer1.get(bytes1, point, SIZE);
// Your logic here
point += SIZE;
}
When you do the above, SIZE vallue increments enourmously and you try to access the byte array with point position which will have a higher vallue than it's size.
So, your logic to access the array position is wrong. As the error line says, you're accessing and index out of bounds( higher than the limit ).
I hope I could help you.
Related
I need to divide a byte array into 3 parts and process them one by one
The first 120 data array is filled in callback function of a Bluetooth device read request, so it's impossible for me to change
byte[] data = new byte[120];
------
byte[] buf = new byte[40];
for (int i = 0; i < 3; ++i) {
System.arraycopy(data, i * 40, packetBuffer, 0, 40);
processDataStream(buf);
}
If in c/c++, I can use pointer so no need to call copy.
In Java, is arraycopy the best way? is there any more efficient way?
Thanks in advance
In most cases you'd just specify an offset and provide it to your method:
byte[] buf = new byte[40];
for (int i = 0; i < 3; ++i) {
processDataStream(buf, i * 40);
}
Then you only need to apply it:
processDataStream(byte[] buf, int offset) {
for (int i = offset; i < offset + 40; i++) {
...
}
}
I'm working on java project based on analysis network traffic. I created classes which are uploading data from input and output files in pcap extension (I uploaded from them time and packet's size). I have a problem with main logarithm which task is checking a buffer condition. I have to check how many times and how long buffer was empty, how many times was unsuccessful attempt to output packet from the buffer and final buffer size.
I created logic in analysis_buffer method, which sets the time chronological, it's mean: if the output_time is bigger than input_time, output_size is adding to buffer(buffer_size), if input_size is bigger, input_size is subtracting from buffer.
I'm using nested loops, first on input file length, second loop(inside) on output file length. Application is working, but finally output is not correctly, the variables are equal 0(only buffer size is much too big) . I have no idea what else I have to change in code, any suggestions?
import org.jnetpcap.util.PcapPacketArrayList;
public class Buffer {
int buffer_size =0; //initial size the buffer
int failed_attempt =0; //count unsuccessful attempt to output packet from the buffer
double time_empty = 0; //time for which buffer was empty
int count_empty =0; //count for which buffer was empty
public Buffer(int buffer_size) {
this.buffer_size = buffer_size;
}
public Buffer() {
// TODO Auto-generated constructor stub
}
public void analysis_buffer(PcapPacketArrayList packetArrayList, PcapPacketArrayList packetArrayList2, double[] input_time, double[] output_time, int[] input_size, int[] output_size) {
for (int i = 0; i < packetArrayList.size(); i++) {
for (int j = 0; j < packetArrayList2.size(); j++) {
// if output time is greater than input time, add input size to main buffer size
if (input_time[i] < output_time[j]) {
buffer_size = buffer_size + input_size[i];
}
// if input time is greater than output time, we have 3 options
// 1. buffer size is greater than output size packet and output is subtracting from buffer size
// 2. buffer size is smaller than output packet - buffer size is automatically equal 0
// 3. buffer size is equal 0, I'm increasing unsuccessful attempt to get out of the buffer and saves time when buffer was empty
else if (input_time[i] > output_time[j]) {
if(output_size[j] < buffer_size) {
buffer_size = buffer_size - output_size[j];
}
if(output_size[j] > buffer_size) {
buffer_size = 0;
count_empty++;
}
if(buffer_size == 0) {
failed_attempt++;
time_empty = time_empty + (output_time[j+1]-output_time[j]);
}
}
// if input time is equal output time, add and subtract packets from buffer size, or buffer size is automatically equal 0
else if (input_time[i] == output_time[j]) {
if(output_size[j] < buffer_size) {
buffer_size = buffer_size + input_size[i] - output_size[j];
}
if(output_size[j] > (buffer_size + input_size[i])){
buffer_size = 0;
count_empty++;
}
}
}
}
}
public void check_buffer() {
System.out.println("Initial buffer size was 0");
System.out.println("Final buffer size: " + buffer_size +".");
System.out.println("Buffer was empty " + count_empty +" times.");
System.out.println("Failed attempt to output data from the buffer: " + failed_attempt +" times.");
System.out.println("Total time for which the buffer was empty: " + time_empty + " seconds.");
}
}
My output:
Initial buffer size was 0
Final buffer size: 1227700210.
Buffer was empty 1 times.
Failed attempt to output data from the buffer: 0 times.
Total time for which the buffer was empty: 0.0 seconds.
I have tried your code with these values:
String[] p = new String[4];
String[] p2 = new String[4];
double[] it = {5432d, 4234d, 6345d, 64320d, 8534d};
double[] ot = {5436d, 4234d, 6342d, 64326d, 8534d};
int[] is = {45, 654, 79, 16354, 4563};
int[] os = {65, 641, 98, 23346, 9846};
Buffer buffer = new Buffer();
buffer.analysis_buffer(p, p2, it, ot, is, os);
buffer.check_buffer();
And I got this result:
Initial buffer size was 0
Final buffer size: 16354.
Buffer was empty 3 times.
Failed attempt to output data from the buffer: 3 times.
Total time for which the buffer was empty: 62200.0 seconds.
The failed_attempt and time_empty variables are modified only in this place:
if(buffer_size == 0) {
failed_attempt++;
time_empty = time_empty + (output_time[j+1]-output_time[j]);
}
I think that the condition hasn't been met (with your test values), and maybe neither the next:
if(output_size[j] > buffer_size) {
buffer_size = 0;
count_empty++;
}
and
else if (input_time[i] > output_time[j]) {
BTW, try to fix this:
java.lang.ArrayIndexOutOfBoundsException: 5
at Buffer.analysis_buffer(Buffer.java:47)
With these values:
String[] p = new String[5];
String[] p2 = new String[5];
double[] it = {5432d, 4234d, 6345d, 64320d, 8534d};
double[] ot = {5436d, 4234d, 6342d, 64326d, 8534d};
int[] is = {45, 654, 79, 16354, 4563};
int[] os = {65, 641, 98, 23346, 9846};
Buffer buffer = new Buffer();
buffer.analysis_buffer(p, p2, it, ot, is, os);
buffer.check_buffer();
I have searched for a way to resize an array in Java, but I could not find ways of resizing the array while keeping the current elements.
I found for example code like int[] newImage = new int[newWidth];, but this deletes the elements stored before.
My code would basically do this: whenever a new element is added, the array largens by 1. I think this could be done with dynamic programming, but I'm, not sure how to implement it.
You can't resize an array in Java. You'd need to either:
Create a new array of the desired size, and copy the contents from the original array to the new array, using java.lang.System.arraycopy(...);
Use the java.util.ArrayList<T> class, which does this for you when you need to make the array bigger. It nicely encapsulates what you describe in your question.
Use java.util.Arrays.copyOf(...) methods which returns a bigger array, with the contents of the original array.
Not nice, but works:
int[] a = {1, 2, 3};
// make a one bigger
a = Arrays.copyOf(a, a.length + 1);
for (int i : a)
System.out.println(i);
as stated before, go with ArrayList
Here are a couple of ways to do it.
Method 1: System.arraycopy():
Copies an array from the specified source array, beginning at the specified position, to the specified position of the destination array. A subsequence of array components are copied from the source array referenced by src to the destination array referenced by dest. The number of components copied is equal to the length argument. The components at positions srcPos through srcPos+length-1 in the source array are copied into positions destPos through destPos+length-1, respectively, of the destination array.
Object[] originalArray = new Object[5];
Object[] largerArray = new Object[10];
System.arraycopy(originalArray, 0, largerArray, 0, originalArray.length);
Method 2: Arrays.copyOf():
Copies the specified array, truncating or padding with nulls (if necessary) so the copy has the specified length. For all indices that are valid in both the original array and the copy, the two arrays will contain identical values. For any indices that are valid in the copy but not the original, the copy will contain null. Such indices will exist if and only if the specified length is greater than that of the original array. The resulting array is of exactly the same class as the original array.
Object[] originalArray = new Object[5];
Object[] largerArray = Arrays.copyOf(originalArray, 10);
Note that this method usually uses System.arraycopy() behind the scenes.
Method 3: ArrayList:
Resizable-array implementation of the List interface. Implements all optional list operations, and permits all elements, including null. In addition to implementing the List interface, this class provides methods to manipulate the size of the array that is used internally to store the list. (This class is roughly equivalent to Vector, except that it is unsynchronized.)
ArrayList functions similarly to an array, except it automatically expands when you add more elements than it can contain. It's backed by an array, and uses Arrays.copyOf.
ArrayList<Object> list = new ArrayList<>();
// This will add the element, resizing the ArrayList if necessary.
list.add(new Object());
You could just use ArrayList which does the job for you.
It is not possible to change the Array Size.
But you can copy the element of one array into another array by creating an Array of bigger size.
It is recommended to create Array of double size if Array is full and Reduce Array to halve if Array is one-half full
public class ResizingArrayStack1 {
private String[] s;
private int size = 0;
private int index = 0;
public void ResizingArrayStack1(int size) {
this.size = size;
s = new String[size];
}
public void push(String element) {
if (index == s.length) {
resize(2 * s.length);
}
s[index] = element;
index++;
}
private void resize(int capacity) {
String[] copy = new String[capacity];
for (int i = 0; i < s.length; i++) {
copy[i] = s[i];
s = copy;
}
}
public static void main(String[] args) {
ResizingArrayStack1 rs = new ResizingArrayStack1();
rs.push("a");
rs.push("b");
rs.push("c");
rs.push("d");
}
}
You could use a ArrayList instead of array. So that you can add n number of elements
List<Integer> myVar = new ArrayList<Integer>();
Standard class java.util.ArrayList is resizable array, growing when new elements added.
You can't resize an array, but you can redefine it keeping old values or use a java.util.List
Here follows two solutions but catch the performance differences running the code below
Java Lists are 450 times faster but 20 times heavier in memory!
testAddByteToArray1 nanoAvg:970355051 memAvg:100000
testAddByteToList1 nanoAvg:1923106 memAvg:2026856
testAddByteToArray1 nanoAvg:919582271 memAvg:100000
testAddByteToList1 nanoAvg:1922660 memAvg:2026856
testAddByteToArray1 nanoAvg:917727475 memAvg:100000
testAddByteToList1 nanoAvg:1904896 memAvg:2026856
testAddByteToArray1 nanoAvg:918483397 memAvg:100000
testAddByteToList1 nanoAvg:1907243 memAvg:2026856
import java.util.ArrayList;
import java.util.List;
public class Test {
public static byte[] byteArray = new byte[0];
public static List<Byte> byteList = new ArrayList<>();
public static List<Double> nanoAvg = new ArrayList<>();
public static List<Double> memAvg = new ArrayList<>();
public static void addByteToArray1() {
// >>> SOLUTION ONE <<<
byte[] a = new byte[byteArray.length + 1];
System.arraycopy(byteArray, 0, a, 0, byteArray.length);
byteArray = a;
//byteArray = Arrays.copyOf(byteArray, byteArray.length + 1); // the same as System.arraycopy()
}
public static void addByteToList1() {
// >>> SOLUTION TWO <<<
byteList.add(new Byte((byte) 0));
}
public static void testAddByteToList1() throws InterruptedException {
System.gc();
long m1 = getMemory();
long n1 = System.nanoTime();
for (int i = 0; i < 100000; i++) {
addByteToList1();
}
long n2 = System.nanoTime();
System.gc();
long m2 = getMemory();
byteList = new ArrayList<>();
nanoAvg.add(new Double(n2 - n1));
memAvg.add(new Double(m2 - m1));
}
public static void testAddByteToArray1() throws InterruptedException {
System.gc();
long m1 = getMemory();
long n1 = System.nanoTime();
for (int i = 0; i < 100000; i++) {
addByteToArray1();
}
long n2 = System.nanoTime();
System.gc();
long m2 = getMemory();
byteArray = new byte[0];
nanoAvg.add(new Double(n2 - n1));
memAvg.add(new Double(m2 - m1));
}
public static void resetMem() {
nanoAvg = new ArrayList<>();
memAvg = new ArrayList<>();
}
public static Double getAvg(List<Double> dl) {
double max = Collections.max(dl);
double min = Collections.min(dl);
double avg = 0;
boolean found = false;
for (Double aDouble : dl) {
if (aDouble < max && aDouble > min) {
if (avg == 0) {
avg = aDouble;
} else {
avg = (avg + aDouble) / 2d;
}
found = true;
}
}
if (!found) {
return getPopularElement(dl);
}
return avg;
}
public static double getPopularElement(List<Double> a) {
int count = 1, tempCount;
double popular = a.get(0);
double temp = 0;
for (int i = 0; i < (a.size() - 1); i++) {
temp = a.get(i);
tempCount = 0;
for (int j = 1; j < a.size(); j++) {
if (temp == a.get(j))
tempCount++;
}
if (tempCount > count) {
popular = temp;
count = tempCount;
}
}
return popular;
}
public static void testCompare() throws InterruptedException {
for (int j = 0; j < 4; j++) {
for (int i = 0; i < 20; i++) {
testAddByteToArray1();
}
System.out.println("testAddByteToArray1\tnanoAvg:" + getAvg(nanoAvg).longValue() + "\tmemAvg:" + getAvg(memAvg).longValue());
resetMem();
for (int i = 0; i < 20; i++) {
testAddByteToList1();
}
System.out.println("testAddByteToList1\tnanoAvg:" + getAvg(nanoAvg).longValue() + "\t\tmemAvg:" + getAvg(memAvg).longValue());
resetMem();
}
}
private static long getMemory() {
Runtime runtime = Runtime.getRuntime();
return runtime.totalMemory() - runtime.freeMemory();
}
public static void main(String[] args) throws InterruptedException {
testCompare();
}
}
You can try below solution inside some class:
int[] a = {10, 20, 30, 40, 50, 61};
// private visibility - or change it as needed
private void resizeArray(int newLength) {
a = Arrays.copyOf(a, a.length + newLength);
System.out.println("New length: " + a.length);
}
It is not possible to resize an array. However, it is possible change the size of an array through copying the original array to the newly sized one and keep the current elements. The array can also be reduced in size by removing an element and resizing.
import java.util.Arrays
public class ResizingArray {
public static void main(String[] args) {
String[] stringArray = new String[2] //A string array with 2 strings
stringArray[0] = "string1";
stringArray[1] = "string2";
// increase size and add string to array by copying to a temporary array
String[] tempStringArray = Arrays.copyOf(stringArray, stringArray.length + 1);
// Add in the new string
tempStringArray[2] = "string3";
// Copy temp array to original array
stringArray = tempStringArray;
// decrease size by removing certain string from array (string1 for example)
for(int i = 0; i < stringArray.length; i++) {
if(stringArray[i] == string1) {
stringArray[i] = stringArray[stringArray.length - 1];
// This replaces the string to be removed with the last string in the array
// When the array is resized by -1, The last string is removed
// Which is why we copied the last string to the position of the string we wanted to remove
String[] tempStringArray2 = Arrays.copyOf(arrayString, arrayString.length - 1);
// Set the original array to the new array
stringArray = tempStringArray2;
}
}
}
}
Sorry, but at this time is not possible resize arrays, and may be never will be.
So my recommendation, is to think more to find a solution that allow you get from the beginning of the process, the size of the arrays that you will requiere. This often will implicate that your code need a little more time (lines) to run, but you will save a lot of memory resources.
We can't do that using array datatype. Instead use a growable array which is arrayList in Java.
How can I remove the first n number of bytes from a ByteBuffer without changing or lowering the capacity? The result should be that the 0th byte is the n+1 byte. Is there a better data type in Java to do this type of action?
You could try something like this:
public void removeBytesFromStart(ByteBuffer bf, int n) {
int index = 0;
for(int i = n; i < bf.position(); i++) {
bf.put(index++, bf.get(i));
bf.put(i, (byte)0);
}
bf.position(index);
}
Or something like this:
public void removeBytesFromStart2(ByteBuffer bf, int n) {
int index = 0;
for(int i = n; i < bf.limit(); i++) {
bf.put(index++, bf.get(i));
bf.put(i, (byte)0);
}
bf.position(bf.position()-n);
}
This uses the absolute get and put method of the ByteBuffer class and sets the position at next write position.
Note that the absolute put method is optional, which means that a class that extends the abstract class ByteBuffer may not provide an implementation for it, for example it might throw a ReadOnlyBufferException.
Whether you choose to loop till position or till limit depends on how you use the buffer, for example if you manually set the position you might want to use loop till limit. If you do not then looping till position is enough and more efficient.
Here is some testings:
#Test
public void removeBytesFromStart() {
ByteBuffer bf = ByteBuffer.allocate(16);
int expectedCapacity = bf.capacity();
bf.put("abcdefg".getBytes());
ByteBuffer expected = ByteBuffer.allocate(16);
expected.put("defg".getBytes());
removeBytesFromStart(bf, 3);
Assert.assertEquals(expectedCapacity, bf.capacity());
Assert.assertEquals(0, bf.compareTo(expected));
}
#Test
public void removeBytesFromStartInt() {
ByteBuffer bf = ByteBuffer.allocate(16);
int expectedCapacity = bf.capacity();
bf.putInt(1);
bf.putInt(2);
bf.putInt(3);
bf.putInt(4);
ByteBuffer expected = ByteBuffer.allocate(16);
expected.putInt(2);
expected.putInt(3);
expected.putInt(4);
removeBytesFromStart2(bf, 4);
Assert.assertEquals(expectedCapacity, bf.capacity());
Assert.assertEquals(0, bf.compareTo(expected));
}
I think the method you are looking for is the ByteBuffer's compact() method
Even though the documentation says:
"The bytes between the buffer's current position and its limit, if any, are copied to the beginning of the buffer. That is, the byte at index p = position() is copied to index zero, the byte at index p + 1 is copied to index one, and so forth until the byte at index limit() - 1 is copied to index n = limit() - 1 - p. The buffer's position is then set to n+1 and its limit is set to its capacity."
I am not sure that this method realy does that, because when I debug it seems like the method just does buffer.limit = buffer.capacity.
Do you mean to shift all the element to the begining of the buffer? Like this:
int n = 4;
//allocate a buffer of capacity 10
ByteBuffer b = ByteBuffer.allocate(10);
// add data to buffer
for (int i = 0; i < b.limit(); i++) {
b.put((byte) i);
}
// print buffer
for (int i = 0; i < b.limit(); i++) {
System.out.print(b.get(i) + " ");
}
//shift left the elements from the buffer
//add zeros to the end
for (int i = n; i < b.limit() + n; i++) {
if (i < b.limit()) {
b.put(i - n, b.get(i));
} else {
b.put(i - n, (byte) 0);
}
}
//print buffer again
System.out.println();
for (int i = 0; i < b.limit(); i++) {
System.out.print(b.get(i) + " ");
}
For n=4 it will print:
0 1 2 3 4 5 6 7 8 9
4 5 6 7 8 9 0 0 0 0
Use compact method for that. E.g.:
ByteBuffer b = ByteBuffer.allocate(32);
b.put("hello,world".getBytes());
b.position(6);
b.compact();
System.out.println(new String(b.array()));
Hello how to convert String into int array. I need it for some enrypting.
Just a String converted into 32 bit values.
I tried that but it did not work. Maybe converting String into BigInteger then this into raw String then into int array would work?
String s = "Alice";
int[] tab = s.getBytes();
I think something like this would work for ya: Found it here: http://pro-programmers.blogspot.com/2011/05/java-byte-array-into-int-array.html
public int[] toIntArray(byte[] barr) {
//Pad the size to multiple of 4
int size = (barr.length / 4) + ((barr.length % 4 == 0) ? 0 : 1);
ByteBuffer bb = ByteBuffer.allocate(size *4);
bb.put(barr);
//Java uses Big Endian. Network program uses Little Endian.
bb.order(ByteOrder.BIG_ENDIAN);
bb.rewind();
IntBuffer ib = bb.asIntBuffer();
int [] result = new int [size];
ib.get(result);
return result;
}
To call it:
String s = "Alice";
int[] tab = toIntArray(s.getBytes());
If you are converting a String to an int array please read Joel's article on String encoding, it is not as obvious as you may think.
Try :
String s = "1234";
int[] intArray = new int[s.length()];
for (int i = 0; i < s.length(); i++) {
intArray[i] = Character.digit(s.charAt(i), 10);
}
change to byte : byte[] tab = s.getBytes();
final String s = "54321";
final byte[] b = s.getBytes();
for (final byte element : b) {
System.out.print(element+" ");
}
Output :
53 52 51 50 49
Edit
(int) cast is removed by Eclipse in System.out.print((int) element+" ");
Unless you want to cast int myInteger = (int) tab[n], you have to copy byte[] in a new int[]
String s = "Alice";
byte[] derp = s.getBytes();
int[] tab = new int[derp.length];
for (int i=0; i< derp.length; i++)
tab[i] = (int)derp[i];
You can't convert a string to bytes without an encoding.
You need to use this already existing method:
public static final byte[] getBytesUtf8( String string )
{
if ( string == null )
{
return new byte[0];
}
try
{
return string.getBytes( "UTF-8" );
}
catch ( UnsupportedEncodingException uee )
{
return new byte[]
{};
}
}
}
Then change it to int array like this: byte[] bAlice
int[] iAlice = new int[bAlice.length];
for (int index = 0; index < bAlice.length; ++index) {
iAlice [index] = (int)bAlice[index];
}
The rules of widening primitive conversion does not apply to arrays of primitive types. For instance, it is valid to assign a byte to an integer in Java:
byte b = 10;
int i = b; //b is automatically promoted (widened) to int
However, primitive arrays do not behave the same way, and therefore, you cannot assume that an array of byte[] is going to be automatically promoted to an array of int[].
However, you could force the promotion of every item in your byte array manually:
String text = "Alice";
byte[] source = text.getBytes();
int[] destiny = new int[source.length];
for(int i = 0; i < source.length; i++){
destiny[i] = source[i]; //automatically promotes byte to int.
}
For me, this would be the simplest approach.