java.lang.OutOfMemoryError during String concatenation - java

I get java.lang.OutOfMemoryError during String concatenation. Can somebody help me get rid of this? Below is how my code looks. This whole if blokc is run under a loop and when it is processing string concatenation for "str", it throws OutofemoryError. Any help on this much appreciated.
for (long j = mincollectiontime; j <= maxcollectiontime; j = j
+ timeintreval) {
query = "select count(*) table1";
ResultSet result2 = VerticaDBHandler.executequery(con2, query);
System.out.println("Query:- " + query);
String str = "";
if (result2.isBeforeFirst()) {
if (mysqlconn == null) {
mysqlconn = DatabaseHandler.openDB();
}
while (result2.next()) {
int isgap = Integer.parseInt(result2.getString(1));
if (isgap == 0) {
Date startime = EpochTimeHandler.epochToTimeStamp(j);
Date endtime = EpochTimeHandler.epochToTimeStamp(j
+ timeintreval);
str = "NO DATA BETWEEN " + startime + " --- " + endtime
+ " forInstanceId: " + instanceid
+ " --InstanceName: " + instanceName + " in "
+ perfTables[i];
DatabaseHandler.LoadDB_dataGaps(mysqlconn,
perfTables[i], instanceid, instanceName,
VirtType, BelongstoDataCenter,
startime.toString(), endtime.toString(), str);
System.out.println(str);
str = "";
} else {
System.out.println("No Gap Seen");
}
}
}
}
The exception.
Exception thrown:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.LinkedHashMap.newNode(LinkedHashMap.java:256)
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.put(HashMap.java:611)
at sun.util.resources.OpenListResourceBundle.loadLookup(OpenListResourceBundle.java:146)
at sun.util.resources.OpenListResourceBundle.loadLookupTablesIfNecessary(OpenListResourceBundle.java:128)
at sun.util.resources.OpenListResourceBundle.handleKeySet(OpenListResourceBundle.java:96)
at java.util.ResourceBundle.containsKey(ResourceBundle.java:1807)
at sun.util.locale.provider.LocaleResources.getTimeZoneNames(LocaleResources.java:262)
at sun.util.locale.provider.TimeZoneNameProviderImpl.getDisplayNameArray(TimeZoneNameProviderImpl.java:122)
at sun.util.locale.provider.TimeZoneNameProviderImpl.getDisplayName(TimeZoneNameProviderImpl.java:98)
at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getName(TimeZoneNameUtility.java:325)
at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(TimeZoneNameUtility.java:281)
at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(TimeZoneNameUtility.java:267)
at sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObjectImpl(LocaleServiceProviderPool.java:281)
at sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObject(LocaleServiceProviderPool.java:265)
at sun.util.locale.provider.TimeZoneNameUtility.retrieveDisplayName(TimeZoneNameUtility.java:135)
at java.util.TimeZone.getDisplayName(TimeZone.java:400)
at java.util.Date.toString(Date.java:1045)
at java.lang.String.valueOf(String.java:2982)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at com.test.perf.testpgm.main(testpgm.java:112)

Seems like here is the Problem.
for (long j = mincollectiontime; j <= maxcollectiontime; j = j + timeintreval)
You are definitely ruling out the maximum lengh and giving a call in the function. Please post more code so that, specific problem can be noticed.

May be this issue is not related to string concationation, I think this issue with Heap size. Once increase the heap size of your IDE and run the program.

Related

How do I properly print enumerator elements?

I am trying to understand java program written by someone else and I do not know java. I have written a short method fro dumping attributes of request object.
public void dumpRequest(HttpServletRequest request) {
String[] attrNames = new String[100]; // hard coded
int ani = 0;
Enumeration rns = request.getAttributeNames();
while (rns.hasMoreElements()) {
out.println("attribute name: " + rns.nextElement());
attrNames[ani] = rns.nextElement().toString();
ani = ani + 1;
}
out.println("" + ani + " atributes");
String cn;
for (int n = 0; n < ani; n++) {
cn = attrNames[n];
out.println("** " + cn + " - " + request.getAttribute(cn));
}
out.println("++++++++++++++++++++++");
}
To my horror, I have realised that NetBeans variables tab shows twice more attributes on the request object compared to my code output.
The enumeration seems to be documented here:
https://tomcat.apache.org/tomcat-4.1-doc/catalina/docs/api/org/apache/catalina/util/Enumerator.html
What am I doing wrong?
You call nextElement method twice in this block:
while (rns.hasMoreElements()) {
out.println("attribute name: " + rns.nextElement());
attrNames[ani] = rns.nextElement().toString();
ani = ani + 1;
}
you should call nextElement once. Put it in variable and then use that variable.

Why is my jpql .getResultList() returning 0 rows for a good query

I was using the exact same query yesterday and it was working fine today I made a few changes to flow of the program and the query no longer returns and rows.
the first function that my programs goes to:
public void prepareSummary(Date startDate , Date endDate)
{
int getStartDay = getDayFromDate(startDate);
int getStartMonth = getMonthFromDate(startDate);
//
int getEndDay = getDayFromDate(endDate);
int getEndMonth = getMonthFromDate(endDate);
int getYear = getYearFromDate(startDate);
if(getStartMonth <= getEndMonth)
{
if(getStartMonth == getEndMonth)
{
if(getStartDay < getEndDay)
{
while(getStartDay <= getEndDay)
{
Calendar cal = Calendar.getInstance();
cal.set( getYear, getStartMonth, getStartDay);
Date queryStart = getStartOfDay(cal.getTime());
Date queryEnd = getEndOfDay(cal.getTime());
List<Object[]> res = getSumList(queryStart, queryEnd);
doQuery(res);
++getStartDay;
}
}
else
{
}
}
else
{
}
}
else
{
}
}
Here is what getSumList looks like:
public List<Object[]> getSumList(Date start, Date end) {
String query = "";
query += "SELECT COUNT(s) pCount,"
+ "p.nameText,"
+ "g.nameText,"
+ "t.shiftID"
+ " FROM Sheets s , GradeNames g , SpecieNames p, ShiftTimes t"
+ " WHERE s.createdLocal > :start and s.createdLocal < :end"
+ " AND s.specieNameIndex = p.nameIndex "
+ " AND s.gradeNameIndex = g.nameIndex"
+ " AND s.shiftIndex = t.shiftIndex"
+ " GROUP BY p.nameText , g.nameText , t.shiftID";
Query q = em.createQuery(query);
q.setParameter("start", start);
q.setParameter("end", end);
return q.getResultList();
}
This next function doesn't matter at this point because nothing is being executed because the list length is zero:
private void doQuery(List<Object[]> obj)
{
int length = obj.size();
String grade = null;
Long standingCount = (long) 0;
System.out.println("Length" + length);
for (int i = 0; i < length; ++i) {
// HAVE A LIST OF ALL ITEMS PULLED FROM DATABASE
Object[] tmpObj = obj.get(i);
Long tmpCount = (Long) tmpObj[0];
String tmpSpecieName = (String) tmpObj[1];
Double tmpThickness = Double.parseDouble(getSpecie().getThicknessFromSpecie(tmpSpecieName));
String tmpLength = getSpecie().getLengthFromSpecie(tmpSpecieName);
String tmpGradeName = (String) tmpObj[2];
String tmpShift = (String) tmpObj[3];
tmpSpecieName = getSpecie().getSpecieFromSpecie(tmpSpecieName);
//// END OF ALL ITEMS PULLED FROM DATABASE
if (grade != pullGradeName(tmpGradeName) && grade != null) {
System.out.println("Count:" + standingCount + "Grade:" + tmpGradeName + "--" + "Specie" + tmpSpecieName + "Shift:" + tmpShift + "Thickness:" + tmpThickness + "Length:" + tmpLength + "SpecieNAme:" + tmpSpecieName);
// do previous insert
grade = pullGradeName(tmpGradeName);
} else if (grade != pullGradeName(tmpGradeName) && grade == null) {
grade = pullGradeName(tmpGradeName);
} else if (grade == pullGradeName(tmpGradeName)) {
standingCount = standingCount + tmpCount;
}
System.out.println("Count:" + tmpCount + "Grade:" + tmpGradeName + "--" + "Specie" + tmpSpecieName + "Shift:" + tmpShift + "Thickness:" + tmpThickness + "Length:" + tmpLength + "SpecieNAme:" + tmpSpecieName);
}
}
Check the SQL that is generated, and the tables you are querying over. As the query requires inner joins, if one of the tables was cleared, it would return no results. If you want to get a 0 count, you need to use an outer join syntax which isn't possible in JPA unless you use object level mappings:
"SELECT COUNT(s) pCount,"
+ "p.nameText,"
+ "g.nameText,"
+ "t.shiftID"
+ " FROM Sheets s outer join s.specialNameIndex p,"
+ " outer join s.gradeNameIndex g, outer join s.shiftIndex t"
+ " WHERE s.createdLocal > :start and s.createdLocal < :end"
+ " GROUP BY p.nameText , g.nameText , t.shiftID";

java.lang.ArrayIndexOutOfBoundsException :

I have a String = "abc model 123 abcd1862893007509396 abcd2862893007509404", if I provide space between abcd1 & number eg. abcd1 862893007509396 my code will work fine, but if there is no space like abcd1862893007509396, I will get java.lang.ArrayIndexOutOfBoundsException, please help ?:
PFB the code :
String text = "";
final String suppliedKeyword = "abc model 123 abcd1862893007509396 abcd2862893007509404";
String[] keywordarray = null;
String[] keywordarray2 = null;
String modelname = "";
String[] strIMEI = null;
if ( StringUtils.containsIgnoreCase( suppliedKeyword,"model")) {
keywordarray = suppliedKeyword.split("(?i)model");
if (StringUtils.containsIgnoreCase(keywordarray[1], "abcd")) {
keywordarray2 = keywordarray[1].split("(?i)abcd");
modelname = keywordarray2[0].trim();
if (keywordarray[1].trim().contains(" ")) {
strIMEI = keywordarray[1].split(" ");
for (int i = 0; i < strIMEI.length; i++) {
if (StringUtils.containsIgnoreCase(strIMEI[i],"abcd")) {
text = text + " " + strIMEI[i] + " "
+ strIMEI[i + 1];
System.out.println(text);
}
}
} else {
text = keywordarray2[1];
}
}
}
After looking at your code the only thing i can consider for cause of error is
if (StringUtils.containsIgnoreCase(strIMEI[i],"abcd")) {
text = text + " " + strIMEI[i] + " "
+ strIMEI[i + 1];
System.out.println(text);
}
You are trying to access strIMEI[i+1] which will throw an error if your last element in strIMEI contains "abcd".

(Java + Android) Parsing string to float strange error

This may sound like a trivial question, but I'm having a really hard time trying to figure it out. Basically I'm sending a string from my Android to my PC. All the connection is ok, and the string is transfered successfully. This is the Android code (sends string to computer):
try
{
println(scSocket + "");
if (scSocket!=null)
{
SendReceiveBytes sendReceiveBT = new SendReceiveBytes(scSocket);
String red = rotZ + " \n";
byte[] myByte = stringToBytesUTFCustom(red);
sendReceiveBT.write(myByte);
}
}
catch(Exception e)
{
println(e);
}
Where rotZ is what I want to send, it is a "float" value. I need to put the " \n" on the end of the message so that it will be recognized as a full message on the PC. So far so good. Now I want to read this on my PC, which is achieved by:
//BlueTooth
String lineRead = "";
try
{
lineRead = new String(sampleSPPServer.readFromDevice());
if(lineRead != null && !lineRead.isEmpty())
{
String lineTransf = lineRead.replace("\n", "").replace("\r", "").replace(" ", "").replace("\"", "").trim();
println("LineTransf: " + lineTransf);
rotZ += 0.01*(Float.parseFloat(lineTransf));
println("Zrotation: " + rotZ); //Never gets here, throws and error before...
}
else
rotZ += 0;
}
catch(Exception e)
{
println("Exception: " + e);
}
Which gives me the error:
NumberFormatException: invalid float value: "1.1400002"
In my code you can see I check for null, empty, etc. So that's not the problem. I've already tried:
NumberFormat nf = NumberFormat.getInstance(Locale.US);
rotZ += 0.01*(nf.parse(lineTransf).floatValue());
Got the same result... In stackoverflow there is a similar question:
Here
There is one more strange thing, If I try the code:
for(int i = 0; i < lineTransf.length(); i++)
println(lineTransf.substring(i,1));
I get that the string's length is 19, but it only prints the first two and gives the message:
Exception: java.lang.StringIndexOutOfBoundsException: String index out of range: -1
Even more strange thing, when I did ctrl-c, ctrl-v on the number "1.1400002" that appears in the console, it only pastes "1" here on stack overflow.
I know that the number is right, but somewhere the conversion is not. I think that's because the string is sent as a byte and read as a String, but how do I solve this problem? Thanks in advance!!
Nothing strange. that's the expected behavior of substring. It throws an IndexOutOfBoundsException, if the startIndex is negative, the endIndex is greater than the string's length or if startIndex is greater the endIndex (which is your case). To me it looks like you want to print the char at index. Try with
for(int i = 0; i < lineTransf.length(); i++)
println(lineTransf.charAt(i));
I found a work around, but I really, really would like an explanation (if possible), because this is just too ugly... I changed the code to:
//BlueTooth
String lineRead = "";
try
{
lineRead = new String(sampleSPPServer.readFromDevice());
if(lineRead != null && !lineRead.isEmpty())
{
String lineTransf = lineRead.replace("\n", "").replace("\r", "").replace(" ", "").replace("\"", "").trim();
println("LineTransf: " + lineTransf + " " + lineTransf.length());
String lastTry = "";
for(int i = 0; i < lineTransf.length(); i++)
{
if(lineTransf.charAt(i) != ' ' && lineTransf.charAt(i) != '\u0000')
{
println(lineTransf.charAt(i));
lastTry += lineTransf.charAt(i);
}
}
println("LastTry: " + lastTry);
rotZ += 0.01*(Float.parseFloat(lastTry));
println("Zrotation: " + rotZ);
}
else
rotZ += 0;
//System.out.println("Line Read:" + lineRead);
}
catch(Exception e)
{
println("Exception: " + e);
}
I'm basically creating a new String called lastTry and then checking if each of the bluetooth read characters are not empty(?) null(?) (since I'm testing for:)
if(lineTransf.charAt(i) != ' ' && lineTransf.charAt(i) != '\u0000')
And if they pass this test I individually "assemble" the lastTry String. It seems that the bluetooth is sending a null character between each of the characters of the whole string. I don't understand why this happens and it actually consumes some time while reading the incoming string. I really would love another answer if someone have another idea...

Memory management in java when running sql in a loop

I have been facing some trouble with memory management. I have some code here that runs an sql query in a loop, puts the data into Array Lists and then does some computations. I have run many similar programs before without this problem. The reason I put the query in a loop was so that too much memory wouldn't be stored in java objects at once. However, now when I run the program, I get a memory error at the exact same place every time (when it is at the 29th iteration of the loop).
Here is the error -
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Arrays.copyOf(Unknown Source)
at java.util.ArrayList.grow(Unknown Source)
at java.util.ArrayList.ensureCapacityInternal(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at transnMat.bootTrnsn.main(bootTrnsn.java:82)
I have pasted the code below, I'd really appreciate any tips on what I might change to get rid of this -
Connection conn = null;Statement st = null;ResultSet rstru = null;
for(int i=start;i<stop;i++) {
double[][] forvariance = new double[(demos.length-1)][numsims];
ArrayList<Long> hhids1 = new ArrayList<>();
ArrayList<Double> outlierwt = new ArrayList<>();
ArrayList<String> fbdemos = new ArrayList<>();
ArrayList<String> trudemos = new ArrayList<>();
rstru = st.executeQuery(
"select TRUTH_DEMO_ID, FB_DEMO_ID, RN_ID, OUTLIER_WEIGHT from SCRATCH.." +
months + "monthtable where BRAND_ID = " + brands[i] +
" order by RN_ID");
while (rstru.next()) { //Get query results and put them into a hash map.
String temp0 = rstru.getString(1);
String temp1 = rstru.getString(2);
String temp2 = rstru.getString(3);
String temp3 = rstru.getString(4);
//String temp5 = rstru.getString(6);
hhids1.add(Long.parseLong(temp2.substring(0,11)));
fbdemos.add(temp1);
trudemos.add(temp0);
outlierwt.add(Double.parseDouble(temp3));
}
for(int sim=0;sim<numsims;sim++) {
trnsnpv = new double[demos.length][demos.length-1];
HashMap<Long,Integer> thissampl = bootsampl2.get(sim);
for(int i1=0;i1<fbdemos.size();i1++) {
if(thissampl.containsKey(hhids1.get(i1)))
trnsnpv[dems.get(fbdemos.get(i1))][dems.get(trudemos.get(i1))-1] +=
outlierwt.get(i1)*(double)thissampl.get(hhids1.get(i1));
}
for(int j=0;j<trnsnpv.length;j++) { //27 rows
trnsnpv[j] = normalize(trnsnpv[j]);
for(int k=0;k<trnsnpv[j].length;k++) { //26 columns
forvariance[k][sim] += trnsnpv[j][k];
}
}
}
for(int k = 0; k < (demos.length - 1); k++) {
double d = StdStats.var11(forvariance[k]);
fileIO.fileIO.write2file(brands[i] + "," + demos[k+1] +
"," + String.valueOf(d) + "\n", "vars.csv");
}
System.out.println("Brands processed: " + String.valueOf(i-start) +
" out of: " + (stop-start));
hhids1.clear();
outlierwt.clear();
fbdemos.clear();
trudemos.clear();
}
Several performance problems here:
The database has to recompile the query each time because the SQL is not parameterized. Consider the use of a prepared statement.
Nested loops. I see one point where you have 4 nested loops.
There is no way I can figure out what your logic is doing due to the variable names and excessive looping. If it's possible, and not sure if it is with your logic (depends on what aggregation you are doing), can you do everything one object at a time in your while (rs.next()) loop?
Ex:
while (rs.next()) {
String temp0 = rstru.getString(1);
String temp1 = rstru.getString(2);
String temp2 = rstru.getString(3);
String temp3 = rstru.getString(4);
//String temp5 = rstru.getString(6);
// do all of your work in here, so that your objects
// can be garbage collected before the next iteration
}
Here is the version of the code that did work (for my own reference)..
for(int i=start;i<stop;i++){
double[][] forvariance = new double[(demos.length-1)][numsims];
trnsnpv = new double[numsims][demos.length][demos.length-1];
int size = 0;
Long hhids1;
Double outlierwt;
String fbdemos;
String trudemos;
rstru = st.executeQuery("select TRUTH_DEMO_ID, FB_DEMO_ID, RN_ID, OUTLIER_WEIGHT from SCRATCH.."+months+"monthtable where BRAND_ID = " + brands[i]+" order by RN_ID");
while (rstru.next()) {//Get query results and put them into a hash map.
String temp0 = rstru.getString(1);String temp1 = rstru.getString(2);String temp2 = rstru.getString(3);String temp3 = rstru.getString(4);
hhids1 = (Long.parseLong(temp2.substring(0,11)));
fbdemos = (temp1);
trudemos = (temp0);
outlierwt = (Double.parseDouble(temp3));
for(int sim=0;sim<numsims;sim++){
HashMap<Long,Integer> thissampl = bootsampl2.get(sim);
if(thissampl.containsKey(hhids1))
trnsnpv[sim][dems.get(fbdemos)][dems.get(trudemos)-1] += outlierwt*(double)thissampl.get(hhids1);
}
size++;
}
System.out.print("Processing: " + size + " rows");
for(int sim=0;sim<numsims;sim++){
for(int j=0;j<trnsnpv[sim].length;j++){//27 rows
trnsnpv[sim][j] = normalize(trnsnpv[sim][j]);
for(int k=0;k<trnsnpv[sim][j].length;k++){//26 columns
forvariance[k][sim] += trnsnpv[sim][j][k];
}
}
}
for(int k = 0; k < (demos.length - 1); k++){
double d = StdStats.var11(forvariance[k]);
fileIO.fileIO.write2file(brands[i] + "," + demos[k+1] + "," + String.valueOf(d) + "\n", "vars.csv");
}
System.out.print("Brands processed: " + String.valueOf(i-start + 1 ) + " out of: " + (stop-start) + "\n");
//hhids1.clear();outlierwt.clear();fbdemos.clear();trudemos.clear();
}

Categories

Resources