how to speed up my ArrayList searching? - java

I currently have an ArrayList holding objects of a class I have created, I then parse through the ArrayList in a for loop searching and comparing some data from the ArrayList and some global variables that are loaded else where, however this ArrayList is constantly growing and will eventually have about 115 elements to it towards the end, which then takes a very long time to search through, the function that does this is also called once for every line I read from a text file and the text file will usually be around 400-500 lines long so as you can tell it is very slow process even when testing on small files. Is there a way to speed this up by maybe using another collection instead of an ArrayList, my reasoning for using the ArrayList is I have to know what index it is on when it finds a match.
Here is the class:
private ArrayList<PanelData> panelArray = new ArrayList<PanelData>(1);
public class PanelData {
String dev = "";
String inst = "";
double tempStart = 0.0;
double tempEnd = 0.0;
}
Function:
public void panelTimeHandler (double timeStart, double timeEnd) throws SQLException {
PanelData temps = new PanelData();
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
boolean flag = false;
if(!flag)
{
panelArray.add(temps);
flag = true;
}
for(int i = 0; i < panelArray.size(); ++i ) {
if(panelArray.get(i).dev.equals(devIDStr) && panelArray.get(i).inst.equals(instanceStr)) {
if(panelArray.get(i).tempStart <= timeStart && panelArray.get(i).tempEnd >= timeEnd ) {
//Do Nothing
}
else
{
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
insert();
panelArray.set(i, temps);
}
}
else
{
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
panelArray.add(temps);
insert();
}
}
}
If there is something more you would like to see just ask, thanks. Beef.
Update: Added insert() function
private void insert() throws SQLException
{
stmt = conn.createStatement();
String sqlStm = "update ARRAY_BAC_SCH_Schedule set SCHEDULE_TIME = {t '" + finalEnd + "'} WHERE SCHEDULE_TIME >= {t '" + finalStart + "'} AND" +
" SCHEDULE_TIME <= {t '" + finalEnd + "'} AND VALUE_ENUM = 0 AND DEV_ID = " + devIDStr + " and INSTANCE = " + instanceStr;
int updateSuccess = stmt.executeUpdate(sqlStm);
if (updateSuccess < 1)
{
sqlStm = "insert into ARRAY_BAC_SCH_Schedule (SITE_ID, DEV_ID, INSTANCE, DAY, SCHEDULE_TIME, VALUE_ENUM, Value_Type) " +
" values (1, " + devIDStr + ", " + instanceStr + ", " + day + ", {t '" + finalStart + "'}, 1, 'Unsupported')";
stmt.executeUpdate(sqlStm);
sqlStm = "insert into ARRAY_BAC_SCH_Schedule (SITE_ID, DEV_ID, INSTANCE, DAY, SCHEDULE_TIME, VALUE_ENUM, Value_Type) " +
" values (1," + devIDStr + ", " + instanceStr + ", " + day + ", {t '" + finalEnd + "'}, 0, 'Unsupported')";
stmt.executeUpdate(sqlStm);
}
if(stmt!=null)
stmt.close();
}
Update:
Thank you to Matteo, I realized I was adding to the array even if I didnt find a match till the 10th element it would then added to the array the first 9 times which created many extra elements in the array, which was why it was so slow, I added some breaks and did a little tweaking in the function, and it improved the performance a lot. Thanks for all the input

you can use LinkedHashSet. It seems you add only elements to the end of the list, which is exactly what LinkedHashSet does as well, when inserting an element.
Note however, a LinkedHashSet will not allow duplicates, since it is a set.
Searching if an element exists will be O(1) using contains()
Using the LinkedHashSet will also allow you to keep track of where an element was added, and iterating it will be in order of insertion.

What about using a hashmap?
I would create a small class for the key:
class Key {
String dev, instr;
// todo: implements equals & hashCode
}
and create the map:
Map<Key, PanelData> map = new HashMap...
then you can easily find the element you need by invoking map.get(new Key(...)).
Instead of creating a new class, you could also tweak the PanelData class, implementing methods equals & hashcode so that two classes are equal iff their dev and instr are equal. In this case, your map becomes:
Map<PanelData, PanelData> map ...
// to add:
map.put(temps, temps)
// to search:
PanelData elem = map.get(new PanelData(desiredDev, desiredInstr));

Quite a few optimiztions here.
1) the call: panelArray.get(i) is used repeatedly. Declare a PanelData variable outside the loop, but initialize it only once, at the very begining of the loop:
PanelData pd = null;
for (int i = 0; i < panelArray.size(); ++i) {
pd = panelArray.get(i);
...
}
2) If your dataset allows it, consider using a few maps to help speed look up times:
HashMap<String, PanelData> devToPanelDataMapping = new HashMap<String,PanelData>();
HashMap<String, PanelData> instToPanelDataMapping = new HashMap<String,PanelData>();
3) Consider hashing your strings into ints or longs since String.equals() is slow compared to (int == int)
4) If the ArrayList will be read only, perhaps a multithread solution may help. The thread that reads lines from the text file can hand out individual lines of data to different 'worker' threads.

1) Create PanelArray with the max expected size + 10% when you first create it.
List<PanelData> panelArray = new ArrayList<PanelData>(130) - this will prevent dynamic reallocations of the array which will save processing time.
2) What does insert() do? Odds are that is your resource hog.

This problem might best be solved with a different data structure such as a HashMap or SortedSet.
In order to use a HashMap, you would need to define a class that can produce a hash code for the dev and inst string pairs. One solution is something like:
public class DevAndInstPair
{
private String dev, inst;
#Override
public int hashCode() {
return ((dev.hashCode() * 0x490aac18) ^ inst.hashCode());
}
#Override
public boolean equals(Object o) {
if (o == null || !(o instanceof DevAndInstPair)) {
return false;
}
DevAndInstPair other = (DevAndInstPair) o;
return (dev.equals(other.dev) && inst.equals(other.inst));
}
}
You would then use HashMap<DevAndInstPair, PanelData> as the map type.
Alternatively, if you know that a certain character never appears in dev strings, then you can use that character as a delimiter separating the dev value from the inst value. Supposing that this character is a hyphen ('-'), the key values would be dest + '-' + inst and the key type of the map would be String.
To use SortedSet, you would either have PanelData implement Comparable<PanelData> or write a class implementing Comparator<PanelData>. Remember that the compare operation must be consistent with equals.
A SortedSet is somewhat trickier to use than a HashMap, but I personally think that it is the more elegant solution to this problem.

Related

Sorting list of vector clocks (total order)?

I understand that vector clocks only provide a partial order. So you can't directly sort them. For this reason you use a tie-breaker for vectors that are concurrent, resulting in a total order.
However sorting the vector clocks so that every cause comes before every effect in the resulting list doesn't seem to work and I don't entirely get why.
I have extensive tests that show me that comparing two vectors works:
#Override
public int compareTo(VectorClock<K> that) {
var res = 0;
if (this.isAfter(that))
res = 1;
else if (that.isAfter(this))
res = -1;
else
res = this.timestamp.compareTo(that.timestamp);
System.out.println("compare " + this + " : " + that + " => " + res);
return res;
}
public boolean isAfter(VectorClock<K> that) {
boolean anyClockGreater = false;
var set = new HashSet<K>();
set.addAll(this.keySet());
set.addAll(that.keySet());
for (K key : set) {
final Clock thatClock = that.get(key);
final Clock thisClock = this.get(key);
if (thisClock == null || thisClock.isBefore(thatClock)) {
return false;
} else if (thisClock.isAfter(thatClock)) {
anyClockGreater = true;
}
}
// there is at least one local timestamp greater or local vector clock has additional timestamps
return anyClockGreater || that.entrySet().size() < entrySet().size();
}
However when sorting a list of vector clocks, e.g. one with two vectors that have a happenedBefore relationship and a third vector that is concurrent to both others, it may happen that only the concurrent one is compared to the two others, and the vectors that depend on each other are not compared to each other. Instead their order is (wrongly) decided transitively by the tie-breaker:
VectorClock<String> v1 = VectorClock.fromString("{0=23, 1=28, 2=15, 3=23, 4=15, 5=22, 6=14, 7=19}"); // after v3
VectorClock<String> v2 = VectorClock.fromString("{0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8}");
VectorClock<String> v3 = VectorClock.fromString("{0=15, 1=19, 2=15, 3=20, 4=15, 5=22, 6=14, 7=19}"); // before v1
var s = new ArrayList<>(List.of(v1, v2, v3));
s.sort(VectorClock::compareTo);
assertTrue(s.indexOf(v3) < s.indexOf(v1));
Prints (and fails):
compare {0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8} : {0=23, 1=28, 2=15, 3=23, 4=15, 5=22, 6=14, 7=19} => 1
compare {0=15, 1=19, 2=15, 3=20, 4=15, 5=22, 6=14, 7=19} : {0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8} => 1
What is the underlying reason for this? Is this generally impossible or is there an error?

Input/output in GLPK for Java

I find a lot of GLPK for Java examples about how to specify the model (problem/constraints) to the solver and read parameters from a data file, but very little about programmatic parameter input/output.
In my case I need to submit values (array of weights and values) to a knapsack problem programmatically and postprocess the solution as well (perform addtional numeric checks on the solution found) in order to decide whether to proceed or not.
Think of the equivalent of reading a param: line from a data file without calling glp_mpl_read_data or printing details of a solution to a file without calling glp_print_mip/sol/itp.
Can you provide example code or point me to the right resource?
This is only a partial answer. I managed to solve the output part using the
GLPK.get_ipt_obj_val
GLPK.get_mip_obj_val
GLPK.get_ipt_col_val
GLPK.get_mip_col_val
functions as in the following example
static void writeMipSolution(glp_prob lp) {
String name = GLPK.glp_get_obj_name(lp);
double val = GLPK.glp_mip_obj_val(lp);
System.out.println(name + " = " + val);
int n = GLPK.glp_get_num_cols(lp);
for (int i = 1; i <= n; i++) {
name = GLPK.glp_get_col_name(lp, i);
val = GLPK.glp_mip_col_val(lp, i);
System.out.println(name + " = " + val);
}
}
Still investigating the input part, though.

Looping through multiple ArrayLists

I had asked the same question in another forum, but dint get any suitable answers...so Im posting it here. I have the following program:
public void execute(){
public static ArrayList<Long> time = new ArrayList<Long>();
public static ArrayList<Integer> state = new ArrayList<Integer>();
public static ArrayList<Integer> cpu = new ArrayList<Integer>();
for(int i=0; i<time.size(); i++){
if(cpu.get(i).equals(get)){
Long next_time = time.get(i);
Integer next_func = state.get(i);
Integer next_proc = cpu.get(i);
if(next_time.equals(g) && (next_func.equals(test1.func_num))){
Integer func_next = stt.get(i+1);
if(func_next.equals(0)||(func_next.equals(next_func))) {
System.out.println("here");
}
else
System.out.println("here");
if(cpu.get(i+2).equals(get))
if(stt.get(i+2).equals(func_next) || (stt.get(i+2).equals(0)))
System.out.println(stt.get(i+2));
}
}
}
What I want to do is this: I get the value of time, cpu and state from the user. find the match in the arraylist for the corresponding values, then I want to loop through the arraylists for only those values which match the 'cpu'. All the ArrayLists are of same size and contain values corresponding to each other at any given index. How can I do this?
Example:
The ArrayLists contain various values as follows:
time = 1 cpu = 12 state = 24
time = 2 cpu = 12 state = 4
time = 5 cpu = 13 state = 23
time = 6 cpu = 13 state = 26
time = 8 cpu = 11 state = 34
time = 11 cpu = 12 state = 54
time = 13 cpu = 12 state = 56
time = 14 cpu = 11 state = 58
time = 15 cpu = 15 state = 46
This is the situation. And I get value from the user as time=2 cpu=12 state =4....I find the match and after that I want to look for all values corresponding to cpu=12 only..
Base more on the description then code example
You get a input in form of time, cpu and state form user.
You want to find match for those input criteria.
To be able to do that easily, You should create a type for that.
public class Data {
private final int cpu;
private final long time;
private final int state;
public Data(int cpu, long time, int state) {
this.cpu = cpu;
this.time = time;
this.state = state;
}
//add implementation for equals and hashcode methods.
}
The equals and hash code method are responsible to define unique value for object. So when you create an object with the same input the should generate same hashcode.
The you create your collection with those elements
Set<Data> storage = new HashSet<Data>();
in this storage, you should store all data that you want to execute search on.
The search is simple. You create a search item
Data searchItem = new Data(user.getCpu(), user.getTime(), user.getState());
if(storage.contains(searchItem)) {
// action on true
} else {
// action on false
}
Implementing hash code
EDIT:
Q: How to perform on all items for given CPU ?
To support such operation you must have in your code a structure that can deliver you some sort of data based on decision. Typically for this operation is used type Map. This type allow to gather under a key reference to value. The value can be a collection of objects.
Map> dataMap = new HashMap<>();// Java diamond notation.
or you can use [Multimap] from guava.
When you find the match, you do this:
//once you have the index in a Integer var called myVal
Set<Integer> indexes = new HashSet<Integer>();
for(int i=0; i<time.size(); i++){
if (cpu.get(i) == myVal) {
indexes.add(i);
}
}
Now you can use the set of indexes:
for (Integer index: indexes) {
//do whatever
}
This is O(time.size()). Hope this helps
Java is pure oo language. It means not only you must write in Object Oriented Style, but also think everything as objects like real world. Before finding how to solve this problem, I would like to advise you that you should read carefully OOP and Connections framework in Java.
Something like this should work:
bool matchFound = false;
for (int i = 0; i < time.size(); i++) {
long thisTime = time.get(i);
int thisState = state.get(i);
int thisCpu = cpu.get(i);
if (matchFound) {
if (thisCpu == userCpu) {
System.out.println("Time: " + thisTime + " "
+ "State: " + thisState + " "
+ "Cpu: " + thisCpu);
}
} else {
matchFound = (thisTime == userTime
&& thisState == userState
&& thisCpu == userCpu);
}
}

relation between Arraylist and ArrayIndexOutOfBoundsException

What is the relation between List and Array. Why i am getting ArrayIndexOutOfBoundsException.?
Basically i am passing one argument to a method, based on that using HQL, i am returning one set of ArrayList. Now my problem is that. when this ArrayList returns more than 0 (size) this is working exactly the way i want.
But when it returns 0 (size) ArrayList, i am getting Exception. why is that. can any body explain to me
Now i got one more doubt like if an arraylist returns 5(first) & 0(second as size i tested up to) i am not getting any Exception. but when it returns like 200 or more elements its getting exception. why is that? Is there any specific constant no. like up to this many elements array shouldn't give IndexOutofBounds and things like that? can any body explain to me?
here is my code:
public void setUpDisplay()
{
if (_flatView || _expired || _issue){ // these are all my views.
_deptBalances = fetchBalances(null);
}
else if (_searchGen.getGenericName() != null){
_storeGenList.clear();
_deptBalances = fetchBalances(_searchGen.getGenericName());
if (!ListUtil.nullOrEmptyList(_deptBalances)){
// i am displaying the result here.
}
else {
displaying error message.
}
}
// here i am using my _deptBalances to display(i am just putting this list into displaygroup).
}
// here is my method.
public List<DEPTStoreBalance> fetchBalances(String genName){
EntityManager em // Creating instance to entity manager.
List <DEPTStoreBalance> tempList = ListUtil.list();
String expClause = new String();
if (_expired){
expClause = "and gg.bSubjectToExpiration=true " +
"and msb.expirationDate <= :expDate ";
if(_expiringOnly){
expClause = expClause.concat(" and msb.expirationDate > :today");
}
else {
expClause = expClause +
"and (msb.expirationDate > :today " +
"or (balance.qtyBalance > 0 or balance.qtyInTransit > 0)) ";
}
}
else {
expClause = "and ((gg.bSubjectToExpiration=true " +
"and msb.expirationDate > :expDate) " +
"or gg.bSubjectToExpiration=false) ";
if (_flatView || _issue){
expClause = expClause.concat("and (balance.qtyBalance > 0 or balance.qtyInTransit > 0) ");
}
else if (genName != null){
expClause = expClause.concat("and gg.genericName = :genName ");
}
}
String hql = "select balance from DEPTStoreBalance as balance " +
" "+ // here are my joins with multiple tables.
"where dsg.store = :store " +
expClause;
if (_issue)
hql = hql.concat(" and dsi.deptIssue = :deptIssue");
Query q = em.createQuery(hql);
q.setParameter("store", _store); // here i am selecting the store(which is being changing in search component).
if (_issue)//Only saleable items should be issued
q.setParameter("expDate",12 months);
else
q.setParameter("expDate",_minExpDate ); // constant value :3
if (_expired)
q.setParameter("today", new Date());
if (genName != null){
q.setParameter("genName", genName);
}
if (_issue)
q.setParameter("deptIssue", true);
try{
tempList = (List <DEPTStoreBalance>) q.getResultList();
}
catch (NoResultException nre){
//do something
}
finally {
em.close();
}
return tempList;
}
Arraylist is a dynamic array. Array has fixed size where as Arraylist is not.
Example:
if you declare array as
int[] arr = new int[5];
you have to provide the size of an array.
ArrayList al = new ArrayList();
if(al.size()>0) {
// do your things
}
An ArrayList is nothing but a dynamically growing array.
You're experiencing an ArrayIndexOutofBoundException because when you receive an empty list and you're trying to access a particular location which is non-existent, JVM throws you an exception complaining that you've crossed the bound of the internal array a.k.a ArrayIndexOutOfBoundException.
For example, in your case, when you've an empty ArrayList, the size of the list would be 0. However if you try to access an index which is >= 0, JVM will throw an ArrayIndexOutofBoundException
ArrayList internally uses array to store its content. ArrayList is basically sort of dynamic array.
If you try to access an element at index which is not available, you will get ArrayIndexOutofBoundsException.
If your List has, say, 3 members and you are calling list.get(5) the exception is thrown. It is because the ArrayList is implemented by Java array.

cant sort the following tree map

for (a = 0; a < filename; a++) {
Map<Double,String> m = new HashMap<Double,String>();
String pre = "abc";
String post = ".txt";
for (int ii = 0; ii < 11; ii++) {
m.put(similarityScore[a],pre + a + post + '\n');
}
SortedSet<Double> set = new TreeSet<Double>(m.keySet());
for (Double d : set) {
System.out.println(d + " " + m.get(d));
}
}
Output :
0.5773502691896258 abc0.txt
0.5773502691896258 abc1.txt
0.5773502691896258 abc2.txt
NaN abc3.txt
0.5773502691896258 abc4.txt
NaN abc5.txt
NaN abc6.txt
NaN abc7.txt
NaN abc8.txt
0.5773502691896258 abc9.txt
NaN abc10.txt
This code should be able to sort the double values. But it displays the output on top. What happen ?
The problem is almost certainly NaN.
This is, as the name suggests, not a realy number, and behaves very strangely in terms of comparisons. Is NaN greater than, equal to, or less than 0.5773502691896258? It could be any of those results, and isn't even required to be consistent within a single execution of the program. NaN is not even equal to itself, which says something about how preconceptions of the laws of equality, and strong ordering, go out of the window when NaN is involved.
So the fix is not to use a non-numeric and expect Double.compareTo() to do what you want with it. Depending on what NaN means when returned from similarityScore(), there are several approaches you could take. If it means that it's not a match at all, you could have that method return a Double (rather than a double), return null in these cases, and then only add non-null results to the map. If these results should be displayed anyway, then perhaps you could use a result of 0.0 or -1.0, assuming that's less than any "real" similarity score. If you want something more finessed, then returning something as pure and straightforward as a primitive double is likely going to be the problem, and you may need to return your own (simple) domain class instead.
As an aside - why on earth do you create and populate a HashMap, then use a TreeSet to get the iteration order over the keys? If you simply create m as a TreeMap<Double, String> you get exactly the iteration order you want, so can just iterate overm.entrySet()`. It's clearer, more idiomatic (thus more understandable), and more efficient, so there's no reason not to do this.
for (int ii = 0; ii < 11; ii++) {
m.put(similarityScore[a],pre + a + post + '\n');
}
This puts the same value into the map 11 times - you're not referencing ii inside the loop.
for (Double d : set) {
System.out.println(d + " " + m.get(d));
}
This prints the single entry in the map.
You do the above for values 0..filename - Adding a value to the map several times, then printing it and restarting with a new map.
Map<Double,String> m = new HashMap<Double,String>();
for (a = 0; a < filename; a++) {
String pre = "abc";
String post = ".txt";
m.put(similarityScore[a],pre + a + post + '\n');
}
SortedSet<Double> set = new TreeSet<Double>(m.keySet());
for (Double d : set) {
System.out.println(d + " " + m.get(d));
}
This creates a map, populates it with values for 0..filename, then prints it sorted. You'll still have issues with NaN which isn't really sortable.
Map<Double,String> m = new TreeMap<Double,String>();
for (a = 0; a < filename; a++) {
String pre = "abc";
String post = ".txt";
m.put(similarityScore[a],pre + a + post + '\n');
}
for (Double d : m.keySet()) {
System.out.println(d + " " + m.get(d));
}
And this uses a TreeMap - No need for the intermediate Set
For any Collection to sort, the type of the value on which you are sorting should be same. And should implement comparable interface.
In your case you have NaN and Double values to sort.
Your loop means you're sorting for each filename separately. You'll need to pull the sorting out of the loop to get those values sorted. (Ooops, #Eric beat me to it.)

Categories

Resources