Develop Reference

Dynamic Multidimensional Java Structure to Replace Static Multidimensional Array

I am working on refactoring a small portion of an open source large-scale configuration management system for my University.
We're using some open source tools for machine learning like Weka, and the aspect I am assigned to refactor is dealing with data mining and constructing rules.
The open source files we've been using from Liverpool and Japan are working well, but there are some memory usage issues when we use the program on large scale projects.
I've isolated the major memory hogs and come to the conclusion I need to figure out a different data structure to store and manipulate the data. As it stands now, the program is using what end up becoming very large multidimensional arrays of integers, objects, strings, etc.
There are several methods that simply reconfigure the set up of the associations after we are deriving rules for behaviors. In many cases, we are only adding or subtracting a single element, or simply flattening the multidimensional arrays.
I primarily program in C/C++ in general, so I am not an expert on the data structures available in Java. What I am looking to replace the static arrays with is a dynamic structure that can be easily resized without having to create a second multidimensional array.
What is happening now is we are having to create an entirely new structure every time we add and remove rules, objects, or other miscellaneous data from the multidimensional array. Then we are immediately copying into the new array.
I'd like to be able to simply use the same multidimensional array and simply add a new row and column. Subsequently, I'd like to be able to manipulate the data in the structure by simply saving a temporary value and overwriting previous values, shifting left, right, etc.
Can anyone think of any data structures in Java that would fit the bill?
On a related note, I have looked into explicit garbage collection, but have found I can only really suggest the JVM collect by calling System.Gc(), or by manipulating the garbage collection behavior of the JVM by way of tuning. Is there a better or more effective way?
Regards,
Edm

If you have a lot of nulls/zeroes/falses/empty-strings in your matrix, then you can save space by using a sparse matrix implementation. Matrix-toolkits has several sparse matrices that you can use / modify to suit your needs, or you can just use a hashmap with an {x, y} tuple as the key. (The hashmap also has the advantage that there are several external hashmap implementations available, e.g. BerkeleyDB, so that it's unlikely that you'll run out of memory.)

To replace static arrays with a dynamic structure use an ArrayList that grows with data automatically. To have a two-dimensional data structure use a List of List as
List<List<Integer>> dataStore = new ArrayList<List<Integer>>();
dataStore.add(new ArrayList<Integer>());
dataStore.add(Arrays.asList(1, 2, 3, 4));
// Access [1][3] as
System.out.println(dataStore.get(1).get(3)); // prints 4
Since, you touched upon having control over garbage collection (which Java actually does a pretty good job of all by itself) it seems memory management is of paramount importance as this is what's causing the re-factoring in the first place.
You could look into the Flyweight GoF pattern that focuses on sharing of objects instead of repeating them to cut down on the memory footprint of the application. To enable sharing flyweight objects need to be made immutable.
Psuedo code:
// adding a new flyweight obj at [2][1]
fwObjStore.get(2).set(1, FWObjFactory.getInstance(fwKey));
public class FWObjFactory {
private static Map<String, FWObject> fwMap = new HashMap<String, FWObject>();
public static getInstance(String fwKey) {
if (!fwMap.containsKey(fwKey)) {
fwMap.put(fwKey, newFwFromKey(fwKey));
}
return fwMap.get(fwKey);
}
private static FWObject newFwFromKey(String fwKey) {
// ...
}
}

I would look into using a "List of Lists". For example, you could declare something like
List<List<Object>> mArray = new ArrayList<List<Object>>();
Any time you need to add a new "row", you could do something like:
mArray.add (new ArrayList<Object>());
Check out the List interface to see what you can do with Lists in Java and which classes implement the interface (or roll your own!).

There's no multidimentional thing in Java.Java has array of arrays.
You can use ArrayList with type parameter as ArrayList
ArrayList<ArrayList<yourType>> myList = new ArrayList<ArrayList<yourType>>();
Also,don't worry about GC..It would collect as and when required..

Why not use two Lists tangled together? Like so:
List<List<String>> rowColumns = new ArrayList<>();
// Add a row with two entries, or columns:
List<String> oneRow = Arrays.asList("Hello", "World!");
rowColumns.add(oneRow);
Also, consider using a Map with entries mapped to Lists.
Garbage Collection should generally never have to be dealt with explicitly in Java. Usually you want to look for memory leaks whenever one occur first. When that happens, look for background threads that don't die as supposed to or strong references in caches. If you want to read some about the latter issue, you can start here and here.

best way to store a list of strings

What would be the best way to store and read a really long string, with each entry is an index for another array?
Right now I have this
String indices="1,4,6,19,22,54,....."
The string has up to hundred of thousand entries, so I think maybe I could use a data structure like Linked List. Does anyone know if it would be faster to use one?

List<String> list = new ArrayList<String>();
list.add("1");
list.add("2");
you need to declare arraylist of type string.Then add to it.

It would depend on what you'll do with the string (the indices) and the corresponding arrays. Also, it will depend on how you're gonna access them.
I'd suggest you first read an overview about the data structures implemented in java, specially in the Collections Framework.
We could give some suggestions, but you'd have to provide us more information, specially those I mentioned in the beginning (what you want, how this data will be stored and accessed, and so on).
For example, if you need to have a fast access to the indexed data, maybe a string isn't even the best approach. Maybe a map would be better. The indexes could be the keys and the indexed arrays could be the values of the map, for example. But this is just a void example, I strongly suggest you give us more information.

I really like using the ArrayList class, which if your comfortable using arrays, ArrayList or any member of the Collections Framework. Would work really well. For what your trying to do.
ArrayList<String> indices = new ArrayList<String>();
indices.add("");

I have similar hunch in my mind , in which I want to like 1k number of strings and parse them (searching purpose to know it contain item or not).
Hence I found instead of using java collection framework - map or set or list
if I store data simply in array and start parsing data using for-loop, it is faster.
You visit this link and see actual output which we calculated in micro seconds.
https://www.programcreek.com/2014/04/check-if-array-contains-a-value-java/
So using simple brute force is winner in case of unsorted array
(normally we have).
But arrays.BinarySearch() is winner if array is sorted.

Array vs array [] for java

I am writing a program that will be heavily reliant on ... something ... that stores data like an array where I am able to access any point of the data at any given time as I can in an array.
I know that the java library has an Array class that I could use or I could use a raw array[].
I expect that using the Array type is a bit easier to code, but I expect that it is slightly less efficient as well.
My question is, which is better to use between these two, and is there a better way to accomplish the same result?

Actually Array would be of no help -- it's not what you think it is. The class java.util.ArrayList, on the other hand, is. In general, if you can program with collection classes like ArrayList, do so -- you'll more easily arrive at correct, flexible software that's easier to read, too. And that "if" applies almost all the time; raw arrays are something you use as a last resort or, more often, when a method you want to call requires one as an argument.

The Array class is used for Java reflection and is very, very, rarely used.
If you want to store data in an array, use plain old arrays, indicated with [], or as Gabe's comment on the question suggests, java.util.ArrayList. ArrayList is, as your comment suggests easier to code (when it comes to adding and removing elements!!) but yes, is slightly less efficient. For variable-size collections, ArrayList is all but required.

My question is, which is better to use between these two, and is there a better way to accomplish the same result?
It depends on what you are trying to achieve:
If the number of elements in the array is known ahead of time, then an array type is a good fit. If not, a List type is (at least) more convenient to use.
The List interface offers a number of methods such as contains, insert, remove and so on that can save you coding ... if you need to do that sort of thing.
If properly used, an array type will use less space. The difference is particularly significant for arrays of primitive types where using a List means that the elements need to be represented using wrapper types (e.g. byte becomes Byte).
The Array class is not useful in this context, and neither is the Arrays class. The choice is between ArrayList (or some other List implementation class) and primitive arrays.

In terms of ease of use, the Array class is a lot easier to code.
The array[] is quite a problem in terms of the case that you need to know
the size of the list of objects beforehand.
Instead, you could use a HashMap. It is very efficient in search as well as sorting as
the entire process is carried out in terms of key values.
You could declare a HashMap as:
HashMap<String, Object> map = new HashMap<String, Object>();
For the Object you can use your class, and for key use the value which needs to be unique.

Is it possible to create an open-ended array?

I was wondering if I could create an array without having to enter a value. I don't fully understand how they work, but I'm doing an inventory program and want my array to be set up in a way that the user can enter products and their related variables until they are done, then it needs to use a method to calculate the total cost for all the products. What would be the best way to do that?

Use an ArrayList.
This will allow you to create a dynamic array.
http://download.oracle.com/javase/1.5.0/docs/api/java/util/ArrayList.html
Here is an example/overview:
http://www.anyexample.com/programming/java/java_arraylist_example.xml

Yes, you can do this. Instead of using a primitive type array, for example new int[10], use something like the Vector class, or perhaps ArrayList (checkout API docs for the differences). Using an ArrayList looks like this:
ArrayList myList = new ArrayList();
myList.add("Item 1");
myList.add("Item 2");
myList.add("Item 3");
// ... etc
In other words, it grows dynamically as you add things to it.

As Orbit pointed out, use ArrayList or Vector for your data storage requirements, they don't need specific size to be assigned while declaration.

You should get familiar with the Java Collections Framework, which includes ArrayList as others have pointed out. It's good to know what other collection objects are available as one might better fit your needs than another for certain requirements. For instance, if you want to make sure your "list" contains no duplicate elements a HashSet might be the answer.
http://download.oracle.com/javase/tutorial/collections/index.html

The other answers already told how to do it right. For completeness, in Java every array has a fixed size (length) which is determined at creation and never changes. (An array also has a component type, which never changes.)
So, you'll have to create a new (bigger) array when your old array is full, and copy the old content over. Luckily, the ArrayList class does that for you when its internal backing array is full, so you can concentrate on the actual business task at hand.

When to use a List over an Array in Java?

In Java, when would it be preferential to use a List rather than an Array?

I see the question as being the opposite-
When should you use an Array over a List?
Only you have a specific reason to do so (eg: Project Constraints, Memory Concerns (not really a good reason), etc.)
Lists are much easier to use (imo), and have much more functionality.
Note: You should also consider whether or not something like a Set, or another datastructure is a better fit than a List for what you are trying to do.
Each datastructure, and implmentation, has different pros/cons. Pick the ones that excel at the things that you need to do.
If you need get() to be O(1) for any item? Likely use an ArrayList, Need O(1) insert()? Possibly a Linked List. Need O(1) contains()? Possibly a Hashset.
TLDR: Each data structure is good at some things, and bad at others. Look at your objectives and choose the data structure that best fits the given problem.
Edit:
One thing not noted is that you're
better off declaring the variable as
its interface (i.e. List or Queue)
rather than its implementing class.
This way, you can change the
implementation at some later date
without changing anything else in the
code.
As an example:
List<String> myList = new ArrayList<String>();
vs
List<String> myList = new LinkedList<String>();
Note that myList is a List in both examples.
--R. Bemrose

Rules of thumb:
Use a List for reference types.
Use arrays for primitives.
If you have to deal with an API that is using arrays, it might be useful to use arrays. OTOH, it may be useful to enforce defensive copying with the type system by using Lists.
If you are doing a lot of List type operations on the sequence and it is not in a performance/memory critical section, then use List.
Low-level optimisations may use arrays. Expect nastiness with low-level optimisations.

Most people have answered it already.
There are almost no good reason to use an array instead of List. The main exception being the primitive array (like int[]). You cannot create a primitive list (must have List<Integer>).
The most important difference is that when using List you can decide what implementation will be used. The most obvious is to chose LinkedList or ArrayList.
I would like to point out in this answer that choosing the implementation gives you very fine grained control over the data that is simply not available to array:
You can prevent client from modifying your list by wrapping your list in a Collection.unmodifiableList
You can synchronize a list for multithreading using Collection.synchronizedList
You can create a fixed length queue with implementation of LinkedBlockingQueue
... etc
In any case, even if you don't want (now) any extra feature of the list. Just use an ArrayList and size it with the size of the array you would have created. It will use an Array in the back-end and the performance difference with a real array will be negligible. (except for primitive arrays)

Pretty much always prefer a list. Lists have much more functionality, particularly iterator support. You can convert a list to an array at any time with the toArray() method.

Always prefer lists.
Arrays when
Varargs for a method ( I guess you are forced to use Arrays here ).
When you want your collections to be covariant ( arrays of reference types are covariant ).
Performance critical code.

If you know how many things you'll be holding, you'll want an array. My screen is 1024x768, and a buffer of pixels for that isn't going to change in size ever during runtime.
If you know you'll need to access specific indexes (go get item #763!), use an array or array-backed list.
If you need to add or remove items from the group regularly, use a linked list.
In general, dealing with hardware, arrays, dealing with users, lists.

It depends on what kind of List.
It's better to use a LinkedList if you know you'll be inserting many elements in positions other than the end. LinkedList is not suitable for random access (getting the i'th element).
It's better to use an ArrayList if you don't know, in advance, how many elements there are going to be. The ArrayList correctly amortizes the cost of growing the backing array as you add more elements to it, and is suitable for random access once the elements are in place. An ArrayList can be efficiently sorted.

If you want the array of items to expand (i.e. if you don't know what the size of the list will be beforehand), a List will be beneficial. However, if you want performance, you would generally use an array.

In many cases the type of collection used is an implementation detail which shouldn't be exposed to the outside world. The more generic your returntype is the more flexibility you have changing the implementation afterwards.
Arrays (primitive type, ie. new int[10]) are not generic, you won't be able to change you implementation without an internal conversion or altering the client code. You might want to consider Iterable as a returntype.